-- Logs begin at Wed 2022-09-28 15:01:24 UTC, end at Wed 2022-09-28 16:14:56 UTC. -- Sep 28 15:28:08.408682 dv-cinder-859316-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 28 15:28:08.416091 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 28 15:28:08 2022] *** Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: nodename: dv-cinder-859316-1-iscsi Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: machine: x86_64 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: clock source: unix Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: pcre jit disabled Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: detected number of CPU cores: 4 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: current working directory: / Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: detected binary path: /usr/bin/uwsgi-core Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: your processes number limit is 47680 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: your memory page size is 4096 bytes Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: detected max file descriptor number: 2048 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: lock engine: pthread robust mutexes Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: thunder lock: enabled Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 28 15:28:08.422437 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 28 15:28:08.459393 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: Python main interpreter initialized at 0x5575664e89c0 Sep 28 15:28:08.459393 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: python threads support enabled Sep 28 15:28:08.459393 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: your server socket listen backlog is limited to 100 connections Sep 28 15:28:08.459393 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: your mercy for graceful operations on workers is 90 seconds Sep 28 15:28:08.459785 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: mapped 403077 bytes (393 KB) for 2 cores Sep 28 15:28:08.459868 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: *** Operational MODE: preforking *** Sep 28 15:28:08.460099 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: *** uWSGI is running in multiple interpreter mode *** Sep 28 15:28:08.460099 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: spawned uWSGI master process (pid: 81591) Sep 28 15:28:08.460305 dv-cinder-859316-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 28 15:28:08.461372 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: spawned uWSGI worker 1 (pid: 81604, cores: 1) Sep 28 15:28:08.461372 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: spawned uWSGI worker 2 (pid: 81605, cores: 1) Sep 28 15:28:08.461372 dv-cinder-859316-1-iscsi devstack@keystone.service[81591]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 28 15:28:12.736852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:28:12.736852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Sep 28 15:28:12.737666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:28:12.737666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Sep 28 15:28:12.747842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:28:12.747842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 28 15:28:12.748322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:28:12.748322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 28 15:28:12.786780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 15:28:12.786958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 15:28:12.787144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] command line args: [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 15:28:12.787409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 15:28:12.787582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ================================================================================ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 15:28:12.787830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.788071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_dir = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.788254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.788431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_source = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.788608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] debug = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.788802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_config_append = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_dir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.790940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.791173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.791413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.791653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.791829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.792002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.792240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.792478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.792706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.792932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.793158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.793383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.793623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.793863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.794129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.794640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.794819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] publish_errors = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.795045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.795269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.795445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.795529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 15:28:12.795684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.795768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 15:28:12.795899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] command line args: [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 15:28:12.795986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] standard_threads = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 15:28:12.796333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ================================================================================ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 15:28:12.796410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_dir = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.796906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_journal = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_json = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_source = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_stderr = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] debug = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_syslog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.797872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.797977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.798066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.798227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.798312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.798497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.798577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.798731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.798809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.798910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_config_append = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.799076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.799288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_dir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.799634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.799879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.799960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.800126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.800254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.800345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.800428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.800532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.800619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.800702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.800799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.800935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.801040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.801178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.801280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.801412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.801510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.801629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.801756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.801842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.802003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.802093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.802241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.802326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.802491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.802587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.802738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.802825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.802989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.803234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.803311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.803464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] publish_errors = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.803550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.803688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.803766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.803913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.803999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.804083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.804146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.804322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.804410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.804553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.804632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.804785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] standard_threads = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.804871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.805015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.805236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.805350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.805601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_journal = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.805786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_json = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_stderr = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.805992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.806071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_syslog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.806144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.806289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:28:12.806910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.807866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.808929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.809917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.810866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.811935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.812957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.813915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.814916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.815619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.815739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.815852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.815957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.816960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.817918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.818865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.819873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.820883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.821909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.822588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.822718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.822846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.822973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.823900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.external = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.824932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.password = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.token = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.825937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.826953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.827991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.828908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.829974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.830886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.831949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.832995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.833910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.834873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.external = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.835905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.password = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.token = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.836945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.837998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.838937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.839984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.840937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.841954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.842882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.843925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.844964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.845911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.846962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.847883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.848942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.849947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.850981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.851929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.852963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.853998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.854993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.855973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.856955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.857949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.858947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.859895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.860932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.861967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.862917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.863912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.864948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.865999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.866927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.867866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.868782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.869924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.870888 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.871928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.872957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.873869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.875573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.875573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.875819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.875945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.876132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.876376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.876650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.876881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.877127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.877392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.877654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.877903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.878153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.878391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.878654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.878889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.879125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.879360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.879596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.879841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.880077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.880321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.880556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.880803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.881039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.881283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.881515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.881755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.881989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:28:12.882160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 15:28:12.901434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.901652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.901873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.907920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.908150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.908377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.910216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.916747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.938582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.938808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.943294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.943796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.991473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:12.991972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.039932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.040153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.041323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.041561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.198895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.199022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.199107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.199232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.199334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.199445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.221786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.222966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.223087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.223172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.223324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.234726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.235124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.247965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.248994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.249130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.249232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.249343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.249446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:28:13.265619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.265825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:28:13.266166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.266363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:28:13.266477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.266633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:28:13.266718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.266902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.266982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.267363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.267459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:28:13.267537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.267612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.267687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.267916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.268174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.276854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.277128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.277395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.277673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.277756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.277986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.278259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.278572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.284572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.285492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.287460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.287737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.288572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.288953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.289902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.290066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.290214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.290351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.290611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.291407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.291675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.313891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.314989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.315361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.315640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.316442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.316727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.320240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.320518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.320797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.321356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.321642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.321953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.327090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.327374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.328280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.328569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.332953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.333242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.333525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.333810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.334358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.334924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.335119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.335399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.335675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.335946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.336213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.350134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.350431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.351473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.351753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.354784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.356081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.357964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.358256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.359276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.359576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.362431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.363720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.365514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.366786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.368688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.369529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.369812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.370087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.370367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.370504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.370685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.370808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.371093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.371377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.371659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.371936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.386208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.386516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.386784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.387044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.387507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.387799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.388067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.388331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.393738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.395159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.395293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.396696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.396782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.397019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.398199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.398505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.399635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.399915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.400242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.400508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.400907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.401272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.401566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.401850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.402137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.402553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.411850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.412140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.412414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.412690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.413369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.413369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.413605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.413687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.413877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.414160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.414473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.414859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.415141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.415429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.429244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.431194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.432681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.434663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.435571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.436975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.437511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.438699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.438976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.439726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.439870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.440833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.441119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.441533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.445610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.447082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.447906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.449355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.449932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.450301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.450612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.450890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.452241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.452640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.452937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.453216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.458553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.458839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.459114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.460971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.461263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.461546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.465877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.466153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.466467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.466717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.466987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.467256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.467528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.467796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.468080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.468351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.468511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:28:13.468639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.468798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.468890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.469929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.470186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.470500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.470783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.471054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.471327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.471596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.471865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.472143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:28:13.503817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5575664e89c0 pid: 81604 (default app) Sep 28 15:28:13.506809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5575664e89c0 pid: 81605 (default app) Sep 28 15:28:13.507157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2787fc-dd04-4755-8b70-6e1a8a7aa966 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.507442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2787fc-dd04-4755-8b70-6e1a8a7aa966 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.507711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2787fc-dd04-4755-8b70-6e1a8a7aa966 None None] PATH_INFO: `/v3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.511850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.320112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.320627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.320736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.324929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:20.330549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.330993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.331139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.403196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: INFO dbcounter [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] Registered counter for database keystone Sep 28 15:28:20.422824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG oslo_db.sqlalchemy.engines [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 15:28:20.429058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writer thread running {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 15:28:20.608081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 15:28:20.608394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 15:28:20.667011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:20.668665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:20.799119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 28 15:28:21.283486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.353787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: INFO dbcounter [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] Registered counter for database keystone Sep 28 15:28:21.375327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG oslo_db.sqlalchemy.engines [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 15:28:21.381417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writer thread running {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 15:28:21.514938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c1501-e285-44ce-936b-382ffc110ae4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.519023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.519635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.519910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.520176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.528432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.579727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 15:28:21.580069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411daa2-dadd-486a-a734-38065491eb8b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.589417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:28:21.595444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.612541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a715be-a4dd-4d84-9cf4-a313943797d5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.617170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.618925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.619215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.619488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] PATH_INFO: `/v3/projects/4c663536db924b2aba9e9bb0e3254711` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.621108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] RBAC: Authorizing `identity:get_project(project_id=4c663536db924b2aba9e9bb0e3254711)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.672879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 15:28:21.673235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d0e6-3590-4425-821a-08777524dea9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.677732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/projects/4c663536db924b2aba9e9bb0e3254711 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:23.839330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19269-973e-43c4-8d5f-8927770444a1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.839896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19269-973e-43c4-8d5f-8927770444a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.839896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19269-973e-43c4-8d5f-8927770444a1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.843947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:23.849463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.849874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.850096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.870764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:23.872260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:23.880015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 28 15:28:24.148559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b19269-973e-43c4-8d5f-8927770444a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.161136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b19269-973e-43c4-8d5f-8927770444a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.165983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.166611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.166896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.167170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.174772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.176541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42a01-106e-408e-aba8-c2db152a17ec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.185300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:28:24.191138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.200406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa11bd-e211-48d3-955e-acc39c3acabc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.204806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.205460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.205752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.206030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.207405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.209245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b76851-2098-4cac-a17d-e31c14b968be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.210825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:26.423567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aa266-99b5-4c29-bb66-fc89c91a9b90 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.424096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aa266-99b5-4c29-bb66-fc89c91a9b90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.424096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aa266-99b5-4c29-bb66-fc89c91a9b90 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.425452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:26.431256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.431668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.431810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.451416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:26.452593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:26.499805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa509f-4b61-41b1-a4f9-17bdf390f9c9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.500243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa509f-4b61-41b1-a4f9-17bdf390f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.500656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa509f-4b61-41b1-a4f9-17bdf390f9c9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.502668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:26.508507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256519a1-63d2-494f-a747-95050f4de32c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.509457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256519a1-63d2-494f-a747-95050f4de32c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.509874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256519a1-63d2-494f-a747-95050f4de32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.540206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:26.551800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-256519a1-63d2-494f-a747-95050f4de32c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81605) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 15:28:26.552184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-256519a1-63d2-494f-a747-95050f4de32c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81605) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 15:28:26.601621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-256519a1-63d2-494f-a747-95050f4de32c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:26.603226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-256519a1-63d2-494f-a747-95050f4de32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:26.703576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:26.816454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.841272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:26.922507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7e28d-4190-47fe-85d8-3124a7e5d8eb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:26.955255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.955873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.956147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.956415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.967238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:26.969510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256519a1-63d2-494f-a747-95050f4de32c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:26.976116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:26.977166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:26.978556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:26.983449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.066444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.074374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256519a1-63d2-494f-a747-95050f4de32c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.076644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.086090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d106d-a001-4a1b-81d9-832d85b94d39 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.113857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.114788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.115074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.115350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.119670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.120267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.120541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.120809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.121468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:27.121774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.123188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.134797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:27.140387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.181661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.183234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.243377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944ce6e-5d9c-4563-9788-8774e532a926 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.277440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.278072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.278348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.278662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.279789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.281241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.283211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 9/16] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:27.286854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 8/17] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:28:27.288227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.292891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.295695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.304017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.388588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc91a0-aa63-4114-b464-8895c700b15c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.391202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983c53f-d994-4c5f-b367-692e575cb521 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.423128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.423755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.424035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.424307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.424984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.425432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.425829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.426106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.426378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.426872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.428432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:27.434232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.445443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.447452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:27.465688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:27.466849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:27.467866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:27.467866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:27.467866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:27.467866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:27.472556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.516343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.529055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.536297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2820a1d-f960-4cdc-ada0-4abe0e425406 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.569066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.569676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.569956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.570226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.571510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.domain.id=default, user.id=a86c62b56d134029a7eefd7d39742f82) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:27.571818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.573257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fef3e9-23f0-4c7c-9912-f684743128c4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.574703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf34a37-5012-477c-bca6-9f90239ee70e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.579846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/role_assignments?user.id=a86c62b56d134029a7eefd7d39742f82&scope.domain.id=default&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:27.610003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.610662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.610971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.611531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.612203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:27.612509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.613943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf696b-0703-4828-99a1-10ed42a1e673 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.634462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:29.949379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d389c-103e-47fa-bd72-2f1652b46ac3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.949888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d389c-103e-47fa-bd72-2f1652b46ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.949963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d389c-103e-47fa-bd72-2f1652b46ac3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.951269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:29.956753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787cabb-fe33-4f57-a3e4-53adba908a5b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.957204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787cabb-fe33-4f57-a3e4-53adba908a5b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.957292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787cabb-fe33-4f57-a3e4-53adba908a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.979380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9787cabb-fe33-4f57-a3e4-53adba908a5b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:29.980566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9787cabb-fe33-4f57-a3e4-53adba908a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:30.070632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:30.168402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506bf8-5781-4088-936f-cac36c7fb4ef None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.168688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506bf8-5781-4088-936f-cac36c7fb4ef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.168942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506bf8-5781-4088-936f-cac36c7fb4ef None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.170687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 13/24] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:30.176471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.176759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.177037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.203556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:30.204670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:30.261626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.299000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 12/25] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:30.482611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59506bf8-5781-4088-936f-cac36c7fb4ef None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.600049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59506bf8-5781-4088-936f-cac36c7fb4ef None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.641943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.642972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.643479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.643939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.661585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:30.668804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:30.669848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:30.670740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:30.673366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 14/26] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:30.680522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.801944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.808969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d381a-4b2e-4830-9e42-8da41a6696d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.814754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.864282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.865470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.866215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.866740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.867669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:30.868189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.869932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a95e11-7e55-4ed0-b808-3bd9fd2ffa24 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.894249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 13/27] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:30.895947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d6c9-9335-49e3-b5f3-ce2bb41b9b7c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.896632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d6c9-9335-49e3-b5f3-ce2bb41b9b7c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.897297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d6c9-9335-49e3-b5f3-ce2bb41b9b7c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.900214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 14/28] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:30.901827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323d6c9-9335-49e3-b5f3-ce2bb41b9b7c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.983131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.020357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ded43e-6249-4c7a-9160-fd2b8ad8487d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.048367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323d6c9-9335-49e3-b5f3-ce2bb41b9b7c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.084232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.084800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.085083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.085360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.087573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.089227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd499e-7d42-4290-91be-7d1ff6772c51 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.091001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 15/29] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:31.092402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93e854-c648-4de9-90f3-9c825f41de9a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.092844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93e854-c648-4de9-90f3-9c825f41de9a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.093262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93e854-c648-4de9-90f3-9c825f41de9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.114430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1d93e854-c648-4de9-90f3-9c825f41de9a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:31.115871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d93e854-c648-4de9-90f3-9c825f41de9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:31.120352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.121207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.121691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.122136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.136145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.138352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:31.159150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:31.160081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:31.160892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:31.160892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:31.160892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:31.162827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 15/30] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:31.163782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.199940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 16/31] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:31.206465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6d7c-5ed3-485a-8fef-8c50b62caae5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.206465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6d7c-5ed3-485a-8fef-8c50b62caae5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.206465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6d7c-5ed3-485a-8fef-8c50b62caae5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.206465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 17/32] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:31.206465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6d7c-5ed3-485a-8fef-8c50b62caae5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.284829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.300221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.389061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6d7c-5ed3-485a-8fef-8c50b62caae5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.392504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bf4e-faa2-4bbd-87c7-3f14d25a4414 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.458945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.459572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.459845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.460119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.460772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:31.461077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.462002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.462679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.462955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.463225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.464358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.465820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447aaaa-b735-4ea0-a548-7209db8dd6c1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.467162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.471269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 16/33] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:31.472763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.473048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.473321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.488578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 18/34] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:31.494055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.514049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:31.515294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:31.623753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 17/35] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:31.625339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.628829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.636506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77780-6f07-49cc-842c-0db8a1761f5e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.704565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.705582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.706100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.706600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] PATH_INFO: `/v3/domains/default/users/a86c62b56d134029a7eefd7d39742f82/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.709627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a86c62b56d134029a7eefd7d39742f82, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.716116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.724438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.776185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9f642-93a5-442a-8609-4d18ff7b5d6d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.821978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.822620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.822902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.823176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.823834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.825331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d5b6-2ab4-4161-b516-237c984fd68a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.868890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 19/36] 10.222.0.60 () {58 vars in 1361 bytes} [Wed Sep 28 15:28:31 2022] PUT /identity/v3/domains/default/users/a86c62b56d134029a7eefd7d39742f82/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:31.870455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.992835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc7b05-80bf-49c2-96b4-518746cf738e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 18/37] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:28:31 2022] POST /identity/v3/projects => generated 301 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda155d-ac80-4d8e-97ce-c20935963e08 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda155d-ac80-4d8e-97ce-c20935963e08 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda155d-ac80-4d8e-97ce-c20935963e08 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 19/38] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7150e-1cf6-4380-a08a-42dbc15efa71 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.005478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7150e-1cf6-4380-a08a-42dbc15efa71 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.006027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7150e-1cf6-4380-a08a-42dbc15efa71 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.006650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 20/39] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:32.007428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7150e-1cf6-4380-a08a-42dbc15efa71 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:32.046494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:32.047156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.047446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.047736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.048455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:32.050466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deac27-9c64-4b1d-a2d2-bbeafc9ff0b7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:32.080478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 20/40] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 15:28:31 2022] POST /identity/v3/roles => generated 219 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:32.082070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a91e-cdae-4ba2-8bfc-1bc03ac294d7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.083172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a91e-cdae-4ba2-8bfc-1bc03ac294d7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.083940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a91e-cdae-4ba2-8bfc-1bc03ac294d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.109915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bec4a91e-cdae-4ba2-8bfc-1bc03ac294d7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:32.111583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bec4a91e-cdae-4ba2-8bfc-1bc03ac294d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:32.128864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:32.160517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7150e-1cf6-4380-a08a-42dbc15efa71 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:32.171951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:32.211713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:32.212463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.212754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.213028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.213866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:32.215566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:32.223203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 21/41] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:32.224134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.225530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.225530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.246648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:32.247796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:32.251069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 21/42] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 15:28:32 2022] POST /identity/v3/roles => generated 225 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:32.328932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 22/43] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:32.560311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:32.610838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:32.692987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:32.718685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc0ba-39e4-4856-bb54-344c37631f41 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:32.754721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:32.755359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.755638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.755913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.757235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:32.759244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:32.761450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 23/44] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:32.766769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:32.779068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca0a7e-f62e-4df0-84ee-68a5ccf2d5e5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:32.814918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:32.815456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.815746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.816024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.816705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:32.818114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b7c9f-45c4-4740-afd6-114563f9b3ce None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:32.835462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 22/45] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 15:28:32 2022] POST /identity/v3/roles => generated 223 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:32.886140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92525e-dc5c-4074-86fd-3b5fb4be077d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:32.926988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:32.927625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:32.927911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:32.928190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:32.928869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:32.931118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fc91-38ca-4210-b0e6-36df88b40fee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:32.950525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:32.974469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 24/46] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:28:32 2022] POST /identity/v3/projects => generated 312 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:34.340059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8fbca-3694-438e-b016-cd960b62cbe9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.340596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8fbca-3694-438e-b016-cd960b62cbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.340596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8fbca-3694-438e-b016-cd960b62cbe9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.341932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 23/47] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:34.347069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.347484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.347702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.366185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:34.368020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:34.432971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 25/48] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:34.703635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8fbca-3694-438e-b016-cd960b62cbe9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:34.732896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:34.807845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8fbca-3694-438e-b016-cd960b62cbe9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:34.842108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:34.842768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.843056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.843335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.854084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:34.859029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:34.859912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:34.860861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 24/49] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:34.865990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.069483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70f81f-6c82-4b5b-904c-10784858a488 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.104878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.113782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.114401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.114715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.114997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.115667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:35.115977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.117385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84287a-2a0b-4682-b680-18460dd0298e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.125683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 26/50] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.131882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.142538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f934f7-d66a-44da-9114-3521437d7cb1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.142824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f934f7-d66a-44da-9114-3521437d7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.143096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f934f7-d66a-44da-9114-3521437d7cb1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.144410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 27/51] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:35.149975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.150261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.150564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.167718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:35.168921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:35.219399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.234258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 28/52] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:35.234912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c8fa-cf2a-4aa2-ab3f-e2850a77f8fd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.271296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.271930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.272218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.272499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.273653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.276964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.281278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 25/53] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.288328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.385758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37389c1a-8ae3-40c4-9221-d4a779f0326f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.402545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.420087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.420721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.421006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.421284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.422397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.423861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.425445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 29/54] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.431762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.502286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.531966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a6406-ca26-46a6-aaa0-2139ffd8f1b8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.565282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.565887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.566176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.566481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.567221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.domain.id=default, user.id=a86c62b56d134029a7eefd7d39742f82) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:35.567532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.569035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.575378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 26/55] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/role_assignments?user.id=a86c62b56d134029a7eefd7d39742f82&scope.domain.id=default&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.602662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4252753-06f0-409a-bb32-6559a00b574e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.639591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.640158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.640442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.640718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.641853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.643363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfe835-87bd-4036-bea2-5b5b47edd78d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.644990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 30/56] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:35.650885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.672046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.757337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccdde0-53f6-4a8d-9a0f-fed63656662e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.791413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.792028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.792312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.792587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.793243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.794747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df399-e996-4329-ab1c-7e67ff3e9718 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.822441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 27/57] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:37.965463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9291-8078-4e53-aec6-2b69528262f4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:37.965976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9291-8078-4e53-aec6-2b69528262f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:37.966058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9291-8078-4e53-aec6-2b69528262f4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:37.967420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 31/58] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:37.972659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:37.973050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:37.973220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:37.991738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:37.993884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:38.055433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:38.060923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 28/59] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:38.335447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9291-8078-4e53-aec6-2b69528262f4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:38.390036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:38.434983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9291-8078-4e53-aec6-2b69528262f4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:38.468061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:38.468685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.468971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.469244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.470346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:38.471833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8eab3-3644-455e-aac3-d7dc199edb14 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:38.473400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 32/60] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:38.479662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:38.578603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b765d-8b23-4df1-89c1-65649239fcb0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:38.611859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:38.612461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.612745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.613020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.613681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:38.615162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d8137-22a4-4f2c-a96b-d0291b71daef None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:38.641400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 29/61] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:38 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:41.893408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d492a3-a9a1-4603-9634-222d1cbf096f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.894452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d492a3-a9a1-4603-9634-222d1cbf096f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.894973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d492a3-a9a1-4603-9634-222d1cbf096f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.896555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 33/62] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:41.902709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f94f8-a593-4324-a5d7-464393a89439 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.903667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f94f8-a593-4324-a5d7-464393a89439 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.904186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f94f8-a593-4324-a5d7-464393a89439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.927915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-011f94f8-a593-4324-a5d7-464393a89439 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:41.930069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-011f94f8-a593-4324-a5d7-464393a89439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:42.007131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.029786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 30/63] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:42.358622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d492a3-a9a1-4603-9634-222d1cbf096f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.474836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e226-192f-4384-aaa5-3f1176a945e5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.475527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e226-192f-4384-aaa5-3f1176a945e5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.476023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e226-192f-4384-aaa5-3f1176a945e5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.476940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d492a3-a9a1-4603-9634-222d1cbf096f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.482672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 31/64] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:42.484296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac715e-8d1d-426d-b624-da6c385ee7e5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac715e-8d1d-426d-b624-da6c385ee7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.485313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac715e-8d1d-426d-b624-da6c385ee7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.507953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-72ac715e-8d1d-426d-b624-da6c385ee7e5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:42.510228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-72ac715e-8d1d-426d-b624-da6c385ee7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:42.510801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.519123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.521019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.521328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.523047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.537988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-79d2a857-8e71-48ad-9910-190f408638dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:42.544964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:42.545816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:42.546615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 34/65] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:42.547464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ed481-9a13-4b79-b108-18d643af3c93 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.547607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ed481-9a13-4b79-b108-18d643af3c93 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.547881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ed481-9a13-4b79-b108-18d643af3c93 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.550448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 35/66] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:42.550448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192304e-db06-463a-a5e8-b1859085b4f9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.550863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192304e-db06-463a-a5e8-b1859085b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.551073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192304e-db06-463a-a5e8-b1859085b4f9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.552375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 36/67] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:42.553313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541700-78d3-44a4-b9a9-e078afdce787 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.553525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541700-78d3-44a4-b9a9-e078afdce787 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.555032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541700-78d3-44a4-b9a9-e078afdce787 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.555572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 37/68] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:42.556405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541700-78d3-44a4-b9a9-e078afdce787 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.589705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 32/69] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:42.590982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ac17-4b4a-4c50-a4b6-3b5c5ec2b2f0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.591269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ac17-4b4a-4c50-a4b6-3b5c5ec2b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.591545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ac17-4b4a-4c50-a4b6-3b5c5ec2b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.609128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fc95ac17-4b4a-4c50-a4b6-3b5c5ec2b2f0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:42.610323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc95ac17-4b4a-4c50-a4b6-3b5c5ec2b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:42.672184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541700-78d3-44a4-b9a9-e078afdce787 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.681079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 33/70] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:42.683788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.684342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.684814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.705081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:42.706948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.707212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:42.707712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.708062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.708403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.709128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:42.709496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.710991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dc6b9-b781-4687-8043-2a9e04ebb519 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.720336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 38/71] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:42.721632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.722000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.722340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.740399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:42.741449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:42.758946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.782886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.790575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 34/72] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:42.791773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.810232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 39/73] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:42.899194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe44e12-a43f-49c1-870a-036b39ca25cb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.937811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.939208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.939701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.940145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.950893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.956456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.958183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.960595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 35/74] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:42.961654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.998910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.052596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449e85-1849-4ad9-aada-1e44f9389db7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.070342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff30-fee9-49c2-9283-035ed8341748 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.091022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.091650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.091928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.092201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.093282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.094756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.096316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 40/75] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:43.097075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.113610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.114755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.115211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.115705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.126997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:43.134780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.135625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:43.136392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:43.138481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 36/76] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.139213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.141517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.200161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a6da-e7e9-4ae3-a931-97c8c2eb88ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.233710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.234312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.234630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.234905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.235815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.243529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743526e-7bf8-41c3-a5eb-00689961c835 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.287162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.287794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.288075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.288350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.298623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:43.302657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:43.303435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.304534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:43.305290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:43.305870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.305944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 37/77] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.305944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.350022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.405332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12590d-0333-4ea0-ac80-ff083719963e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.407393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 41/78] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:43.408271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.445017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.445638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.445923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.446198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.446894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.448355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.469096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 38/79] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:43 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:43.469923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.483202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.532562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027004-52da-4b27-8b78-e36ef5a774cc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.572726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.573271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.573559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.573840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.574558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.574871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.576273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.587637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 42/80] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.588342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.592733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.600311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb725ad-cf85-498c-ba79-466fbe5223e3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.637041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.637881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.638160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.638458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.650603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:43.655941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.656812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:43.657575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 39/81] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.658257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.722810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b1bd8-cfc4-4874-840f-85d587a23031 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.746078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.765688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.766315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.766810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.767099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.767783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.768094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.769483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.781521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 43/82] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.782196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.791835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949fee-4838-4fbb-b878-608edf1daf37 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.831129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.832625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.832911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.833225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.836716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.837903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.842548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 40/83] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.842548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.862226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.911545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6ccd-1830-45df-8b77-cc4535f94624 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.949305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.949542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.949758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.949964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.951142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.952603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.954197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 44/84] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.954997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.989900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9952-e2a0-47d6-b6ae-b36f3af08bb9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.035582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.036979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.037475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.038275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.039545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:44.040064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.042270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.048966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.054399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 41/85] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:44.058552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.075473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8473ca-3222-4183-bf69-a7980ecdd1e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.116016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.116981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.117438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.117885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.119242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.120891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.123107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 45/86] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:44.124051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.134183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.196978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69c171-3112-4622-9b6a-104e964622b8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.232550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.233163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.233450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.233726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.234489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:44.234809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.235443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055d0de-7aa3-477e-adf9-13d6c3cc01df None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.236273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.242369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 42/87] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:44.251085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.288655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.289723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.290205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.290813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.392159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.453476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.455870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c83ae-16c0-47ac-b278-376b8653716d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.457242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.459845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.497892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.501616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.502109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.502677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.504254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.507266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.510337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 43/88] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:44.511645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.634094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8473de-661f-447b-8c2d-efbb929eefa7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.666718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.673427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.674579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.675088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.675560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.697913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 46/89] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:44.698644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.749171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:44.751286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:44.854906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 44/90] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:44.856079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:44.902815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.935204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23db85-9935-4f0d-bd82-d6dc88647fd6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:44.936693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:44.974037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:44.974678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.974678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.974956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.996655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b474c8-4748-4b18-a0de-1677b5e0436d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.047471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.048949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.075161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.076150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.076652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.077121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.078545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.080225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.082092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 45/91] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.083072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.154667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 47/92] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:45.155503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.157935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.198344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe4d8e-bafa-4599-85e2-c0c5e689bdc3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.201691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.243376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.247030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.247621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.248094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.250947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.252682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.254861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 46/93] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.255833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.265364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964ad9-3df0-49c2-a749-db93b639f78a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.300291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.300913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.301200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.301479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.302227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.302585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.304048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.313064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 48/94] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=9f09201f0de74baf927082c77fc97d7b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.313903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.451397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.510207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08674aca-4139-43dd-aac7-3387165ef841 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.513701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8df279-c825-45b3-af69-fb322641957f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.576059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.576681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.576962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.577236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.577974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=a86c62b56d134029a7eefd7d39742f82) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.578282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.579781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.586177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.586838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.587115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.587384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] PATH_INFO: `/v3/projects/620afbd85d1949ad8a29c6241ae3933f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.588086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 49/95] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/role_assignments?user.id=a86c62b56d134029a7eefd7d39742f82&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.588801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=620afbd85d1949ad8a29c6241ae3933f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.596276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671a8a8-c84a-4e31-b54e-516f11628aa6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.598120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 47/96] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/projects/620afbd85d1949ad8a29c6241ae3933f => generated 312 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.604853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.771423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf58a9-473b-4909-80a3-283c47c4f01f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.807537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.815474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.816102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.816378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.816648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.817383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=620afbd85d1949ad8a29c6241ae3933f, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.817690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.819183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a1fe-3a56-4495-b65d-de95e0d4fc30 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.826902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 50/97] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=620afbd85d1949ad8a29c6241ae3933f&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:46.153753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361268a3-d670-4f45-a0d0-61d27f1ce591 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.153753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361268a3-d670-4f45-a0d0-61d27f1ce591 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.153753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361268a3-d670-4f45-a0d0-61d27f1ce591 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.153753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 48/98] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:46.180853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f18b7-ac91-4ac5-8be7-2f873d6dc644 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.181786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f18b7-ac91-4ac5-8be7-2f873d6dc644 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.182263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f18b7-ac91-4ac5-8be7-2f873d6dc644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.207508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-549f18b7-ac91-4ac5-8be7-2f873d6dc644 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:46.209071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-549f18b7-ac91-4ac5-8be7-2f873d6dc644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:46.292353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 51/99] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:46.386448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2507d6-7742-4534-beff-dc840c7fc8ed None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.387068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2507d6-7742-4534-beff-dc840c7fc8ed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.387581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2507d6-7742-4534-beff-dc840c7fc8ed None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.389114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 49/100] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:46.394746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.395042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.395328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.423338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:46.424476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:46.495183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 52/101] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:46.733436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2507d6-7742-4534-beff-dc840c7fc8ed None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.779555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.816478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.854935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2507d6-7742-4534-beff-dc840c7fc8ed None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.891441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.892203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.892492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.892770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.904711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:46.909874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:46.910792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:46.911543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 50/102] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:46.917173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.918594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.028939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57f48e-ed2a-47ec-b68d-60cc161deae2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.082452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39f0d5-ad3e-4540-81f7-13abc30e286f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.084551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.107132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.107787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.108076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.108402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.115150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.117180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.119195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 53/103] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:47.127370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.146612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.147605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.148652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.149133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.149988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.152708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.154327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.170348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 51/104] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:47.176013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.295283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c230-2e2d-445b-ad27-5fc930bf301e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.332365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.347198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.347837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.348125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.348411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.349090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.350620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.400910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 54/105] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:28:47 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:47.410294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2870eb-5efc-4382-8b19-d2fdef7eac83 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.467074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.467835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.468117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.468442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] PATH_INFO: `/v3/users/de2a594f2159496da8a1e82a9628618f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.490641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] RBAC: Authorizing `identity:get_user(user_id=de2a594f2159496da8a1e82a9628618f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.492180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60f3d9-2165-4ee6-8c5c-b2a0f7a3aa75 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.494037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 52/106] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/users/de2a594f2159496da8a1e82a9628618f => generated 289 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:47.500091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.581129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d015fd-d1ae-4268-b908-d77c53149123 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.581431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d015fd-d1ae-4268-b908-d77c53149123 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.581691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d015fd-d1ae-4268-b908-d77c53149123 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.583087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 53/107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:47.599921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.600220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.600471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.642908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a8e02-e283-421c-a56c-ab89c7b72205 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.646745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:47.648556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:47.671013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.718264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.718968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.719254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.719533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.775640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 54/108] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:47.789088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.816147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.817672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.919969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 55/109] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:47.925997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.074695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e97da-6d5f-4d1c-a940-03ba7e3a0bcc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.118367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.119033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.119323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.119603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.120749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.122214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.125178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 55/110] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.130207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.206986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.233048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ad2f0-2c06-4758-ba6b-84e216303004 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.267898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.268524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.268810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.269090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.269835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=de2a594f2159496da8a1e82a9628618f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.270149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.271694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.282549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 56/111] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/role_assignments?user.id=de2a594f2159496da8a1e82a9628618f&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.304022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.323789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.457389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a340206-8af5-4d35-807c-87d4d6a8a0e8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.509106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.509753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.510085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.510370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.527328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:48.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.533799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:48.534588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 56/112] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.543535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.669142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e581591-4bd4-4099-8a57-c10147c2b432 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.705624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.706253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.706590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.706883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.707649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.707876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.709296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5e3b-c2e7-4b27-b5fa-7c59358543fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.719018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 57/113] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.721478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.727650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.731803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477d53-925d-49bd-8cd5-15f7ff4b8c82 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.732096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477d53-925d-49bd-8cd5-15f7ff4b8c82 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.732381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477d53-925d-49bd-8cd5-15f7ff4b8c82 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.733726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 58/114] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:48.738930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.739192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.739474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.795580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:48.796813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:48.935855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.982379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 59/115] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:48.999749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d535f5-8a9b-4653-8ae5-400e2947dfae None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.081180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.082096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.082552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.082972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.084272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.094780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce23a6-e719-4222-a25c-8ec4d10d2a9b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.096917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 57/116] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.103528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.120457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d772b-e597-460f-aa85-6cbdd65e9e4b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.120457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d772b-e597-460f-aa85-6cbdd65e9e4b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.120457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d772b-e597-460f-aa85-6cbdd65e9e4b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.120457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 58/117] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:49.125679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.126136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.126607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.186089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:49.189588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:49.419697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe41ab-3d0d-4310-a725-b558aafe589e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.432698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 59/118] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:49.478827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.482960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.483620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.483899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.484170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.485351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.486844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.490745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 60/119] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.493968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.495908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.648829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06aada-db69-4396-bbe7-8fa79ce0fa6d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.658756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.706926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8759-3931-4163-bc59-1b0fe1b8201b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.720726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.723213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.723213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.723213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.726423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.727908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.750078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 61/120] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:49 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:28:49.775110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.775934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.776317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.776742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.788864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d0e1cd1d-a10c-4d90-88b0-caa93c5aa320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:49.794095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:49.795090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:49.796148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 60/121] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:49.801776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.812412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26cf6-433f-4d66-bd85-2e9aeee3ae0c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.812723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26cf6-433f-4d66-bd85-2e9aeee3ae0c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.812966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26cf6-433f-4d66-bd85-2e9aeee3ae0c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.814367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 61/122] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:49.821165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.821452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.821723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.859623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:49.860890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:49.917116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.985856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 62/123] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:49.986754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.987895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224163-94c3-4668-9147-0934975a5184 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.028876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.029510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.030347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.030651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.031330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.031639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.033625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.046135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 62/124] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.051822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.105174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.196218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ada58e-08c2-4f70-924e-b816dff2c8a4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.212594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc63e5-e160-46cf-92e9-1a1bb993d434 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.217285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.235602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.237242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.238382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.238970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.242616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.245885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.247716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 63/125] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:50.254306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.281482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.282123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.282403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.282695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.308393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:50.313581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:50.314675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:50.315652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:50.324479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 63/126] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:50.334009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.432143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26494d1-e7d2-4b9e-af5d-ee421f68c0c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.473527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.474388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.474832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.475246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.479128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.480837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.482689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 64/127] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:50.488393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.525047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33447c5-4277-4c08-89c9-7235578db996 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.562246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.576300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.577246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.577705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.578178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.579093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.579611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.581170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.590607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 64/128] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.591678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.625506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.712124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec8418-c428-4ded-b7f6-d9099f5ff252 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.725065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e7262-6f49-48c1-8ece-33dc546ff4f5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.775481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.776155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.776439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.776715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/a86c62b56d134029a7eefd7d39742f82/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.779303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, user_id=a86c62b56d134029a7eefd7d39742f82, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.780753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.786705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.787344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.787626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.787903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.803729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:50.806775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 65/129] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:50 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/a86c62b56d134029a7eefd7d39742f82/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:28:50.807650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:50.821657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:50.822671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:50.823468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:50.826100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 65/130] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:50.832771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.928315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.014834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163f45-8b1e-415b-8865-a034d36a38f1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.076963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997368-f35d-4292-a781-a3f393e5dc8b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.083208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.083841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.084122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.084394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.085568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:51.085875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.087318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51d9c8-fdc2-45fc-a0ab-6804a47b923c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.098639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 66/131] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:51.100304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2ca8-14b5-4bc2-8c9d-d6d9369781f5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.101351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2ca8-14b5-4bc2-8c9d-d6d9369781f5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.101842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2ca8-14b5-4bc2-8c9d-d6d9369781f5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.103419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 67/132] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:51.106251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.107601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2ca8-14b5-4bc2-8c9d-d6d9369781f5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.133427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.134503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.134964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.135383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.138317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.140076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03f76b-93e4-4ad6-bddd-c932cc971ce9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.142960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 66/133] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.144883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.145318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.145769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.170743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:51.172574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:51.233109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2ca8-14b5-4bc2-8c9d-d6d9369781f5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.241017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.282543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 67/134] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:51.283410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.292147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.293078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.293357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.293630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.297277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.300447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1314756-adbd-4c11-9254-b326ae21ac9d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.302548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 68/135] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.306578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f759-efa3-47bc-aab9-59f12b0fe21d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.306903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f759-efa3-47bc-aab9-59f12b0fe21d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.307184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f759-efa3-47bc-aab9-59f12b0fe21d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.308703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 69/136] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:51.313614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f759-efa3-47bc-aab9-59f12b0fe21d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.409324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4abd04-42ae-4d0a-9ed5-5b619f16e67a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.427089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.470179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.470848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.471196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.471479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.472738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.474187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6d8bb-6f66-44ef-9356-f52bd1ba1252 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.476104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 68/137] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.477224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.477508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.477784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.517460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:51.518613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:51.539131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f759-efa3-47bc-aab9-59f12b0fe21d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.610666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.611303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.611582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.611854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] PATH_INFO: `/v3/projects/620afbd85d1949ad8a29c6241ae3933f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.614065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] RBAC: Authorizing `identity:get_project(project_id=620afbd85d1949ad8a29c6241ae3933f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.615562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.618541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 70/138] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/projects/620afbd85d1949ad8a29c6241ae3933f => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.619384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.638465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 69/139] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:51.639713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.678979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.767423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c0164-dfda-4dcb-a4c2-3f97e9eb18e3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.818916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1fc1d-4412-4b4a-be8c-193f6630bcab None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.839170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.839619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.839850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.840139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.840425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.847834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70, role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.849248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.863295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.864431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.864997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.865519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.869435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.874657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.878250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 71/140] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:51 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/9f09201f0de74baf927082c77fc97d7b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:51.878978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 70/141] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.880067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.885465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.035383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd151e-1c18-46aa-bdf2-9ac5323a920d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.077324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.078376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.078899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.079382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] PATH_INFO: `/v3/projects/620afbd85d1949ad8a29c6241ae3933f/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.081862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] RBAC: Authorizing `identity:create_grant(project_id=620afbd85d1949ad8a29c6241ae3933f, user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.083633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.104237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 71/142] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:51 2022] PUT /identity/v3/projects/620afbd85d1949ad8a29c6241ae3933f/users/ed4d4992ad9c4cb6b2c992b66fa6cd70/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:52.105437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.156323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.164422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0649e9-cd9b-4a60-9ec1-23e772084076 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.181990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.237358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33546-b8e6-452e-98eb-858eb46eef31 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.239197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.239838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.240116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.240392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.241069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.250680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.302067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 72/143] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:51 2022] POST /identity/v3/users => generated 283 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:52.322361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.323093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.323376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.323653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.336738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:52.342075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:52.343051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:52.343835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 72/144] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:52.349986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.502807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc12fa-eb72-4a0e-92cf-e7dd5770805f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.547820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.548575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.548732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.548973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.549648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:52.549954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.551501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5118c-b880-43cf-b11b-bf63a09678c4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.561214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 73/145] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.566880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.604244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dab33-d91e-4fc0-8de4-ce17c245fe51 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.604607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dab33-d91e-4fc0-8de4-ce17c245fe51 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.604769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dab33-d91e-4fc0-8de4-ce17c245fe51 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.606213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 74/146] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:52.611596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.611894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.612122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.645510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:52.648016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:52.686777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.767872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.773286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb35f-14ce-4ab3-8b9c-8f901a373f06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.774938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 75/147] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:52.838551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.839214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.839570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.839846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] PATH_INFO: `/v3/users/de2a594f2159496da8a1e82a9628618f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.841051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] RBAC: Authorizing `identity:get_user(user_id=de2a594f2159496da8a1e82a9628618f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.842682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.844345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 73/148] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/users/de2a594f2159496da8a1e82a9628618f => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.850157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.965653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd759ad5-f6ef-4c5f-b4f6-ef77d30c2faa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.009000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.010021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.010513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.011018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.012395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.014057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.015978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 76/149] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.022117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.185037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d3601-077f-431a-937a-5382f8604a5d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.202919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.242855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.257350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.258081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.258368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.258671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/de2a594f2159496da8a1e82a9628618f/roles/90c95706f60e4a8794b9fb3547e3b0ff` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.262979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, user_id=de2a594f2159496da8a1e82a9628618f, role_id=90c95706f60e4a8794b9fb3547e3b0ff)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.264454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.294364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 74/150] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:53 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/users/de2a594f2159496da8a1e82a9628618f/roles/90c95706f60e4a8794b9fb3547e3b0ff => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:53.367119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f938-639d-444b-9759-b587b7d14e74 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.403626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.404269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.404613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.404835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.416460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:53.422582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.423500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:53.424288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 77/151] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:53.430494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.466046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.561212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ede86a-963c-46a9-ae16-cd81580ae7c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.605437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.606060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.606341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.606639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.607324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:53.607631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.609026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.621891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 75/152] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.627820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.719928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.743972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6ce00-280d-44d0-b55d-0ccf6af06089 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.785316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.785949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.786294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.786600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.787764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.789255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04492ad7-85a4-4d4e-96b7-31daf42cf212 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.792066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 78/153] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.797867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.844447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f805fe-b0b3-4deb-87bc-0fa2347012c9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.844745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f805fe-b0b3-4deb-87bc-0fa2347012c9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.844913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f805fe-b0b3-4deb-87bc-0fa2347012c9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.846259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 79/154] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:53.853835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.854127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.854423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.876564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:53.877933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:53.937476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.953576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8709c-b10d-48b6-9409-13ff81118f16 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.997077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 80/155] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:54.020964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.021585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.021871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.022141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.106386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.107834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.214320 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.226151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 76/156] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:54.231858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.354634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.379018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a0c1b-a35b-4a91-878d-a1e48450d444 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.424647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.425320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.425597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.425866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.431139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.432614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.434504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 81/157] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.440746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.604272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386e087-050f-4ac3-be09-00bfa9b9d1d6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.642287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.642847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.643130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.643402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.644133 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.644439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.646013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.654107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 77/158] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.676043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.816027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95235093-868e-4043-969c-70aa75dceca3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.854668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.855767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.856255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.856741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.868660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:54.874481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:54.875344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:54.876166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:54.878538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 82/159] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:54.886253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.984374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.045174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc834f2-4641-470e-b75a-bf4779888e8e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.083334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.083956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.084244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.084521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.085746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.086052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.087464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.100989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 78/160] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.106650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.113805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.242730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44530d-41de-4f73-b659-ecfe3d857224 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.295130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.295861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.296118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.296386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] PATH_INFO: `/v3/users/a86c62b56d134029a7eefd7d39742f82` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.297569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] RBAC: Authorizing `identity:get_user(user_id=a86c62b56d134029a7eefd7d39742f82)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.299132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682945e-3915-4810-ac96-6afc8100005c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.301093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 83/161] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users/a86c62b56d134029a7eefd7d39742f82 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.307013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.443108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee443c-ba13-4611-95ab-6a2920e3dcdf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.465488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.484681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.485305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.485586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.485860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.578940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.580339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.685243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bf4a-c6cd-4af6-aa61-4a969b1833bc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.685804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bf4a-c6cd-4af6-aa61-4a969b1833bc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.685804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bf4a-c6cd-4af6-aa61-4a969b1833bc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.687512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 84/162] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:55.695165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.695742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.696241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.712485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 79/163] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:55.721615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.726455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:55.728890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:55.755342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.845227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 85/164] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:55.901294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b298-2166-4dff-a1e9-07518951e502 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.957276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.957943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.958223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.958566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.959854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.961355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.963249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 80/165] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:55.969118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.974437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.093987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacc907-bd5a-4ac3-9125-506ec56cbab0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.140267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.141202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.141687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.142173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.143130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=a86c62b56d134029a7eefd7d39742f82) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:56.143665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.145288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16700-3db7-4592-9b72-577695d7fa79 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.153341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 86/166] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/role_assignments?user.id=a86c62b56d134029a7eefd7d39742f82&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:56.346511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.438891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.452410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d263ea-49bd-4f92-8990-4dd69f6673e3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.452782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d263ea-49bd-4f92-8990-4dd69f6673e3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.453250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d263ea-49bd-4f92-8990-4dd69f6673e3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.455136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 87/167] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:56.460935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.461092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b62c2-0fe2-40c5-9413-a41bb81e80de None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.461623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.462092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.480878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:56.482446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:56.517263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.518167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.518643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.519221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.547441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:56.552328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:56.553210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:56.554624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:56.558718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 81/168] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:56.563795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.619538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.636084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 88/169] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:56.739398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38ca0e-ce9d-4e24-a6d9-aef430870ead None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.788984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.789593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.789869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.790138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.790825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:56.791124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.792491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.806377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 82/170] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:56.812482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.975832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58263d4-a9e4-4eeb-bdf3-90979a77ede9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.028587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.029154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.029433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.029706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.030885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.032447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.034143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 89/171] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.040511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.058489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.143908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.214902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb3060-24d4-441a-8408-8b3edf6c19a9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.228185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.283240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.283815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.284273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.284740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.299706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667de1-ed9a-4ce0-95da-abe6b256b8e6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.380956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.383218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fed12-9be0-4d31-a65b-5f25281fe4fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.402159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.403278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.403772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.404265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.408629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.440793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-aa37deb1-eda1-42ae-bf81-7c4c88d6f5da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:57.459220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:57.460217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:57.460996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:57.461590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 90/172] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:57.467905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539f19-cdb4-46cf-a2a5-fd35f7da68c8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.468186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539f19-cdb4-46cf-a2a5-fd35f7da68c8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.468458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539f19-cdb4-46cf-a2a5-fd35f7da68c8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.469800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 91/173] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:57.470537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539f19-cdb4-46cf-a2a5-fd35f7da68c8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.615942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 83/174] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:57.617449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.617892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.618313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.651381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:57.653593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:57.670658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539f19-cdb4-46cf-a2a5-fd35f7da68c8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.701263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.714735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.716151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.716590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.717198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.718198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.718751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.720924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.732091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 92/175] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.733957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.801746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 84/176] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:57.802652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.867706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.939353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d9d6-8057-4f12-aa27-c8809f13e85e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.966899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216e4a-9d2d-46a1-bd8b-d511e77eb9d9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.985991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.987161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.987449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.987721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.989675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.991207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be22ed-c798-4c57-bd6c-d0f137a034cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.992869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 93/177] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.999264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18dd87-08e8-44b2-973a-cb65b60e4057 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.999644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18dd87-08e8-44b2-973a-cb65b60e4057 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.999929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18dd87-08e8-44b2-973a-cb65b60e4057 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.001634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 94/178] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:58.002921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527435-a6ba-4650-81f5-c743cd5a948c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.003215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527435-a6ba-4650-81f5-c743cd5a948c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.003592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527435-a6ba-4650-81f5-c743cd5a948c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.004965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 95/179] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:58.010136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.010136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.010136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.023449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.024468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.024970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.025439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] PATH_INFO: `/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.026972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] RBAC: Authorizing `identity:get_user(user_id=ed4d4992ad9c4cb6b2c992b66fa6cd70)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.029703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dafde5-5a87-44be-92e3-1b2da9cd2cbd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.031671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 85/180] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/users/ed4d4992ad9c4cb6b2c992b66fa6cd70 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.033161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.033642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.034106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.038648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:58.040055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:58.055759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:58.057294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:58.107086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.131233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 96/181] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:58.132623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.148481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 86/182] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:58.149819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.190996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.273056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bab3f-a8e1-43dc-a77d-b7161ed6f4fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.285711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2baf89-967f-48c6-b473-211af25c8376 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.320380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.321006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.321284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.321553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.335083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.336134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.336699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.337178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.338208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.338797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.340728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb599f-60ce-4e84-a226-3c7fab39ea6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.347887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 97/183] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=9f09201f0de74baf927082c77fc97d7b => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.349570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec57254-4e6e-493f-8a09-49c93404c4ce None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.350101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec57254-4e6e-493f-8a09-49c93404c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.350612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec57254-4e6e-493f-8a09-49c93404c4ce None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.352578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 98/184] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:58.353778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec57254-4e6e-493f-8a09-49c93404c4ce None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.432386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.434715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90bcf0-2a5e-43af-b58d-c1a736519760 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.448171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.474640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.511707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec57254-4e6e-493f-8a09-49c93404c4ce None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.553743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 87/185] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:58.555056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.555344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.556398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.570199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.570875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.571158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.571421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.581556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:58.583133 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:58.586169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:58.591352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.592382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:58.593300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 99/186] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.594126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.659394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 88/187] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:58.660253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.701277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509e750-e22a-480e-962f-320fe11eb151 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.712040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.732226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.740563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.741189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.741468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.741740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.752732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:58.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.758693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 100/188] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.759819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.776796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280981e-2aeb-407b-9e2a-c71bb72d5e7b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.816064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.816710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.816989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.817603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.829241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:58.834208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.835880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:58.836766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 89/189] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.837423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.872413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838637b-b590-4fd8-809f-4927500409a8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.917482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.918113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.918458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.918706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] PATH_INFO: `/v3/projects/620afbd85d1949ad8a29c6241ae3933f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.919954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] RBAC: Authorizing `identity:get_project(project_id=620afbd85d1949ad8a29c6241ae3933f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.922227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.929151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 101/190] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/projects/620afbd85d1949ad8a29c6241ae3933f => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.929151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.959188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3103ebe-4c05-47fc-be98-bde1211f17ac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.966675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.998640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.016353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.017340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.017820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.018313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.019563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.020094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.023362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.034024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8396d-b2fc-4a14-869e-35f09fe9ca16 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.037447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 90/191] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.039108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.067673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.068279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.068556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.069353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.070020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.070331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.071751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.080001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 102/192] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.080826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.188951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.195848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838ac4-98ad-48a2-bb4e-c2aaab06bbd8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.205385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45fbda-1053-4001-b868-9a17430d5174 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.229805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.230344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.230656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.230934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.231667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=620afbd85d1949ad8a29c6241ae3933f, user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.231980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.233444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.239618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 103/193] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/role_assignments?user.id=ed4d4992ad9c4cb6b2c992b66fa6cd70&scope.project.id=620afbd85d1949ad8a29c6241ae3933f&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.240416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.248552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.249190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.249480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.249850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.250594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.250909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.253109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.263085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 91/194] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.264397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.323753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.366719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be127f90-7809-4d58-8b52-7fc33a658d96 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.389719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b58b-dc9c-48f3-b864-41df14012285 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.408022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.408676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.408960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.409236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.410437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.411956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.414457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 104/195] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.414457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.437319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.437840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.437840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.439644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.451139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.461869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.463964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.466792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 92/196] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.467380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.532302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd841a44-f01a-4788-9367-bdd5342de4ec None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.565793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.566547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.566739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.567010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] PATH_INFO: `/v3/users/de2a594f2159496da8a1e82a9628618f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.568235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] RBAC: Authorizing `identity:get_user(user_id=de2a594f2159496da8a1e82a9628618f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.570305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.572035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 105/197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/users/de2a594f2159496da8a1e82a9628618f => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.572823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.577573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff9a05-60f8-4125-b9c7-a115a1e8065c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.586002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.614555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.615208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.615486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.616294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.617480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.618983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.620781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 93/198] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.621491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.681651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.685339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c188334-4613-4847-bea5-7e1f9b14862b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.718659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.719262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.719539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.719810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.720482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.721917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.728759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4a19-ccc4-4b26-b44f-72ed8d3307dc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.743057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 106/199] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:59 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:59.744025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.770101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.770978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.771261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.772017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.824393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.854081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.855620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.861460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268118a0-32d3-4134-9661-e204865b49cf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.899427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.900371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.900853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.901320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.927644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.979627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 94/200] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:59.980553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.983149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.984712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.072399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.101572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd760ff-54d3-4c00-8f1d-9902c0dfab23 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.118174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 107/201] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:00.119093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.150109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.150748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.151022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.151290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.189368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.231386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.232971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.255191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cd3a-5722-4145-821a-180262dc2f24 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.306992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.307874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.308172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.308450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.310271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.312018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.314766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 108/202] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.315373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.333706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.370581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 95/203] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:00.371932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.451511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8263-9fcf-47c7-b252-ba3ca44bd958 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.482620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.496189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.496829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.497180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.497388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.498624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.500070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.506822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abf99b-4959-43e8-aba5-5dcc178747d4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.507340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 109/204] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.507874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.548406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.549395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.550009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.550533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.551505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=0696db276a664e8397d0c3f90e526df8) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.552043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.553829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.561067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 96/205] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/role_assignments?user.id=0696db276a664e8397d0c3f90e526df8&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.562437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.661180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.701539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394ac4-399d-4810-9a3b-9b80960fced8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.746401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.746401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.746401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.746401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.746811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.746811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.746811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 110/206] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.754901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.770349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1106-570e-44f5-a4a5-4d666d034925 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.824836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.825708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.825993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.827008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.827756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b, user.id=de2a594f2159496da8a1e82a9628618f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.828069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.829776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383a410-237e-4a55-b3a9-4aca1ceb3876 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.845949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 97/207] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/role_assignments?user.id=de2a594f2159496da8a1e82a9628618f&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.853463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.903891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1af7-9259-4033-9d86-86f783028791 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.941515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.942052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.942335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.942642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.943383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=0696db276a664e8397d0c3f90e526df8) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.944307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.945781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccadf7-05d5-446e-a479-8fda61012ae1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.953212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 111/208] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/role_assignments?user.id=0696db276a664e8397d0c3f90e526df8&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=9f09201f0de74baf927082c77fc97d7b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:02.094712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41121664-d509-4ebf-a7ff-7935a031e95e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.095258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41121664-d509-4ebf-a7ff-7935a031e95e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.095258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41121664-d509-4ebf-a7ff-7935a031e95e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.096686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 98/209] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:02.102160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0511-f427-4c10-b92f-ea7edd83c6e9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.102566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0511-f427-4c10-b92f-ea7edd83c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.102745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0511-f427-4c10-b92f-ea7edd83c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.125567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-abab0511-f427-4c10-b92f-ea7edd83c6e9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:02.126784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-abab0511-f427-4c10-b92f-ea7edd83c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:02.197786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 112/210] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:02.360330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad33e-7769-4939-a878-66c6ce75fd52 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.360970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad33e-7769-4939-a878-66c6ce75fd52 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.361435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad33e-7769-4939-a878-66c6ce75fd52 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.363006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 99/211] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:02.368548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.369158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.369663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.387986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:02.389447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:02.441843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.463094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 113/212] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:02.464928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaad33e-7769-4939-a878-66c6ce75fd52 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.568402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaad33e-7769-4939-a878-66c6ce75fd52 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.570023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.603010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.603638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.603914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.604184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.614369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:02.619170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:02.620084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:02.620866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 100/213] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:02.626254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.730296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.735952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3bbae-4f92-4fc6-9adc-0e19c0f55f2f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.781163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.781773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.782049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.782319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.783289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:02.783591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.785661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.799140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 114/214] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:02.805704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.851190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb4c2f-fe39-484e-912a-60d97889839f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.866727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.890639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.891670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.892154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.892719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.894238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.896057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.898262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 101/215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:02.903818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.936903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.939960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8e61-4d91-4fb1-a452-09a9513814e8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.975362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.975995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.976266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.976537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] PATH_INFO: `/v3/users/47fbf635972745fcb0092020e0197319` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.989976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=47fbf635972745fcb0092020e0197319)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.992038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24b0fd-92dc-47e4-9662-3cda86d31ba3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.993657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 115/216] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:02 2022] GET /identity/v3/users/47fbf635972745fcb0092020e0197319 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:02.995681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cc10-7b70-4b04-977b-8e580a17d8c1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.997403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cc10-7b70-4b04-977b-8e580a17d8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.998534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cc10-7b70-4b04-977b-8e580a17d8c1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.000016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 116/217] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:03.000894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433cc10-7b70-4b04-977b-8e580a17d8c1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.024317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08860a0e-a70a-40ca-8590-705765b95aec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.058865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.059469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.060063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.060350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.060995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:03.062873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d505d9-c2d8-47ce-a57c-75a361bac813 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:03.092452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 102/218] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:02 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:03.101405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e95e4-91cc-4e4a-bb4a-98b68702fa46 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.101405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e95e4-91cc-4e4a-bb4a-98b68702fa46 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.101405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e95e4-91cc-4e4a-bb4a-98b68702fa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.119621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b10e95e4-91cc-4e4a-bb4a-98b68702fa46 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:03.120870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b10e95e4-91cc-4e4a-bb4a-98b68702fa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:03.124215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433cc10-7b70-4b04-977b-8e580a17d8c1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.126989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.170537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.171560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.172088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.172550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.215677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 103/219] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:03.276970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:03.278817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:03.307198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667bacb-f792-429d-a534-21e5b4ae0523 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.307198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667bacb-f792-429d-a534-21e5b4ae0523 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.307680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667bacb-f792-429d-a534-21e5b4ae0523 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.310089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 104/220] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:03.315092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.315376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.315647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.339891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:03.341096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:03.357145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.402323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 117/221] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:03.408811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.410941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.444438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 105/222] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:03.515557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.523772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c33239-9c59-4bac-afb6-2e4051d84c04 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.583325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.583964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.584273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.584542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.585693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:03.587374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:03.589296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 118/223] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:03.595674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.677791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.689501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03341208-2c6e-46e6-8f06-cb2a4f4d915d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.711165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676b56-192a-4bbb-9886-cccadb71be0b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.733358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.734007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.734580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.734860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.746751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:03.752461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:03.754567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:03.756559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 106/224] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:03.763919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.767666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.768291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.768570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.768847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.769579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=47fbf635972745fcb0092020e0197319) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:03.769885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:03.771360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:03.774516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.792385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 119/225] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/role_assignments?user.id=47fbf635972745fcb0092020e0197319&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:03.812962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.882283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69769b-13fc-45a3-bc3f-df53605c4daf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.929581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.930123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.930405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.930731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.931764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:03.932078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:03.934142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:03.945797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 107/226] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:03.953041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.018943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01781d22-d16a-4d1d-a517-c27c3e546b12 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.042788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.086116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.087895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.088177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.088443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.093101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.102535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d5fa5-b1db-4faa-a303-5f2b4a245ded None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.115792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:04.121126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:04.121994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:04.122861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 120/227] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:04.132584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.169059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674fc-54db-45db-beab-e491f746569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.169690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674fc-54db-45db-beab-e491f746569c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.169961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674fc-54db-45db-beab-e491f746569c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.170225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674fc-54db-45db-beab-e491f746569c None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.172316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b674fc-54db-45db-beab-e491f746569c None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.173819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b674fc-54db-45db-beab-e491f746569c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.176677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 108/228] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:04.181754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674fc-54db-45db-beab-e491f746569c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.332012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517d379-a4f2-4e9f-93ae-6efc28c00a8d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.363313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674fc-54db-45db-beab-e491f746569c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.399449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.400362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.400782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.401194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.403224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:04.403750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.405543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.422106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 121/229] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:04.429370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.432884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.433750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.435205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.435623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.437993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.439626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.445882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 109/230] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:04.453276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.458029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.536562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.622224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c3fe7-7d73-44c7-8afe-f1d1e6c29634 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.636182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230de38a-489e-414a-b0be-4d55b240a7b0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.672664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.673558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.673977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.674486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/0696db276a664e8397d0c3f90e526df8/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.677243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, user_id=0696db276a664e8397d0c3f90e526df8, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.678947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.694697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 110/231] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:04 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/0696db276a664e8397d0c3f90e526df8/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:04.696536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.697173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.697453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.697763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.699089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.700565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.702445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 122/232] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:04.708818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.833304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.889553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc346b-673d-4116-a091-8b7c34d591ec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.945804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.946507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.946798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.947286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.950839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.952625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a5f68-0356-43c1-8a57-1e62c2cfe347 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.955835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 111/233] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:04.961260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.101807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883aa33-9bc3-4eb8-8d35-cbe9d609087b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.134318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.146237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.147009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.147290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.147559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/0696db276a664e8397d0c3f90e526df8/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.152071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, user_id=0696db276a664e8397d0c3f90e526df8, role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.153522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa4235-0477-4371-96b1-dbe92b642d32 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.172267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 123/234] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:04 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/0696db276a664e8397d0c3f90e526df8/roles/9f09201f0de74baf927082c77fc97d7b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:29:05.903751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4238c-5d8d-4665-8e75-6e99834acb1a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.904297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4238c-5d8d-4665-8e75-6e99834acb1a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.904297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4238c-5d8d-4665-8e75-6e99834acb1a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.905944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 112/235] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:05.911633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93054-0ac2-4854-9500-45e356f2bf55 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.911922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93054-0ac2-4854-9500-45e356f2bf55 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.912727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93054-0ac2-4854-9500-45e356f2bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.931358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-64d93054-0ac2-4854-9500-45e356f2bf55 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:05.932707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-64d93054-0ac2-4854-9500-45e356f2bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:06.005386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 124/236] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:06.240754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafcf16-358b-4d28-869b-8e2fb71b690d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.241084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafcf16-358b-4d28-869b-8e2fb71b690d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.241265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafcf16-358b-4d28-869b-8e2fb71b690d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.242809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 113/237] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:06.254216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a9b0-77e5-4578-919d-9a8a78c2c46e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.254788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a9b0-77e5-4578-919d-9a8a78c2c46e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.254788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a9b0-77e5-4578-919d-9a8a78c2c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.284298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3eb3a9b0-77e5-4578-919d-9a8a78c2c46e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:06.285500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3eb3a9b0-77e5-4578-919d-9a8a78c2c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:06.315830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafcf16-358b-4d28-869b-8e2fb71b690d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.389931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 125/238] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:06.391434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2fde6-d665-45e9-b6e7-b00f070ce95b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.391721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2fde6-d665-45e9-b6e7-b00f070ce95b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.391993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2fde6-d665-45e9-b6e7-b00f070ce95b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.393521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 126/239] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:06.398890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.399168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.399436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.443623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:06.444804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:06.450682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafcf16-358b-4d28-869b-8e2fb71b690d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.487263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.491038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.491669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.492065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.492343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.493774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.495591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.497197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 114/240] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:06.506441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.524530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.574166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 127/241] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:06.665560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9719e-3cf2-42db-a211-5279bb1c0f05 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.670974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.705959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.706969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.708017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.708484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.709354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.711277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.744086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 115/242] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:06 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:06.854950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801e487-3c06-4628-9b89-3ada1674cf44 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.908468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.909131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.909410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.909684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.932426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:06.937688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:06.938606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:06.939529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 128/243] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:06.945405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.963431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.023393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.024854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.076103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d91598-5013-406e-89bd-407d02159089 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.116132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.116754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.117031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.117302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.117966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.118264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.119663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.120694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c518d-9d7b-4427-a49f-a2b74dcbb4fa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.134594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 116/244] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.135517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.161253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.162553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.162950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.163230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.175833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:07.183081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:07.183963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:07.185253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 129/245] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:07.194392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.251047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5608d77-15aa-4abe-bc81-e1e680f77087 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.275110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.293509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.294311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.294716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.295078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] PATH_INFO: `/v3/users/47fbf635972745fcb0092020e0197319` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.296536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] RBAC: Authorizing `identity:get_user(user_id=47fbf635972745fcb0092020e0197319)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.297990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.299809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 117/246] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/users/47fbf635972745fcb0092020e0197319 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.305707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.313596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.326126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88979eb-d163-410b-8986-1a0c60185603 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.365898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.366991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.367475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.368012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.368980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.369502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.371396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.381478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 130/247] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.392765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.410583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd128d08-98c6-43bd-8964-b16a99d3931b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.447227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.448292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.448961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.449438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.451292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.453256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.455319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 118/248] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:07.464832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.517608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd24e-4d45-41d0-bb9f-4f3e008b8aeb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.539661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.563993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.565100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.565580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.566219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] PATH_INFO: `/v3/users/c4cdbca6890c478e90a2c143db4c8733` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.588605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cdbca6890c478e90a2c143db4c8733)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.590445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.592180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b79d-fb0f-403c-8308-f6e64300a79f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.592675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 131/249] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/users/c4cdbca6890c478e90a2c143db4c8733 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:07.598490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.616727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.664157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.664846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.665128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.665404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/47fbf635972745fcb0092020e0197319/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.671948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, user_id=47fbf635972745fcb0092020e0197319, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.673480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc627d8-48d5-43bd-9d1c-0a00fe254f62 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.706752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 119/250] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:07 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/47fbf635972745fcb0092020e0197319/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:07.727703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f8edf-306c-4b1b-817e-2e9f133b3f25 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.771612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.772663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.773244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.773714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.794137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313a3ec-3214-4e4f-8f7e-315d82f8f331 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.794434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313a3ec-3214-4e4f-8f7e-315d82f8f331 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.794664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313a3ec-3214-4e4f-8f7e-315d82f8f331 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.796263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 120/251] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:07.802281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f82b2-c344-4977-82e4-ede589f93e66 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.802281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f82b2-c344-4977-82e4-ede589f93e66 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.802281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f82b2-c344-4977-82e4-ede589f93e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.839569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4c0f82b2-c344-4977-82e4-ede589f93e66 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:07.840798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4c0f82b2-c344-4977-82e4-ede589f93e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:07.869515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.871458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.890264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.004733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 132/252] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:08.011472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.037400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 121/253] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:08.052592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a59f1-0b90-40cc-b112-12e830b5391b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.053239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a59f1-0b90-40cc-b112-12e830b5391b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.053453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a59f1-0b90-40cc-b112-12e830b5391b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.055209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 122/254] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:08.061076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.061329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.061951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.082775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:08.084426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:08.155105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.203326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd85f8-cebc-401c-bb8f-0aa11320aa80 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.210636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 123/255] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:08.227513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.256738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.257374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.257652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.257919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.263544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.264677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.266699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 133/256] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:08.272654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.416336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.436351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d191c-f34d-4f6b-8e4a-eec2d7d6e1b3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.493242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.494225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.494766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.495247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.496235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=c4cdbca6890c478e90a2c143db4c8733) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.496746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.498559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.509345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 124/257] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/role_assignments?user.id=c4cdbca6890c478e90a2c143db4c8733&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.605992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371c91-706c-4d39-b370-872d45d5cda3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.631399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.663115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.663763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.664048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.664316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.698539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:08.704889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:08.705874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:08.706743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 134/258] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:08.732813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.735887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.798803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.831235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51681f3-2bee-4fdb-9a2b-ab7b542459dd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.891397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20dd4-712f-410e-82ac-295d788a5f11 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.901981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.902679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.902959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.903289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.920158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:08.929101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:08.930197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:08.931042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 125/259] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:08.936029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.958191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.959106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.959523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.959930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.960781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.961295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.962891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.983581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 135/260] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:08.989338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.114807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.118019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a617-32a3-4db7-853f-a4cdeff4daee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.118997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3fd8c-b270-405b-ab60-d69ffc2ffbcc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.126764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.174915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.175761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.177222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.177543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.179600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.181066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.182976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 136/261] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.191974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.192368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.192593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.192869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.193183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.193852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:09.194151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.195572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.204220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 126/262] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:09.210654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.377025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c909-ee6e-4f0e-8974-99dcf7b82a49 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.386854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0657-40d6-43b7-a760-4d632d23c771 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.437206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.437840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.438080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.438353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.449271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.450188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.450605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.450884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] PATH_INFO: `/v3/users/0696db276a664e8397d0c3f90e526df8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.452047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] RBAC: Authorizing `identity:get_user(user_id=0696db276a664e8397d0c3f90e526df8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.453481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.458523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 127/263] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/users/0696db276a664e8397d0c3f90e526df8 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.464036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.518763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.526918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.564456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.565920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.647025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29d1a-bc02-418b-b531-2d568ea35efb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.711719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.712958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.713433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.713897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.730843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 137/264] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:09.736772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.844433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.846327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.867604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.901677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.939595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14ec54-4a5f-4f89-88fd-08f872d19aaf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.006684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.007238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.007510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.007776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.008972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.010445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.017285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 138/265] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:10.019325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.049822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 128/266] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:10.058339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.186263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647394c4-546d-44ba-acfd-c85d7f504e40 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.245024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.246014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.246517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.246997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.247346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf22c79-cd9f-4f77-9190-637b0474484f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.248584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=0696db276a664e8397d0c3f90e526df8) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:10.249101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.250993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.261592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 139/267] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/role_assignments?user.id=0696db276a664e8397d0c3f90e526df8&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:10.285492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.294630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.295265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.295542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.295816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.296932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.298387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f10e-2bc5-412b-8d25-2004642981a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.305168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 129/268] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.311981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.326803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.463325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79a0ae-1690-4b29-98c7-992d2ab4efd2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.511846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.512374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.512650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.512922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.513660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=0696db276a664e8397d0c3f90e526df8) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:10.513966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.515459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee365-f646-4377-bdf4-c14fe815dee9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.525724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 140/269] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/role_assignments?user.id=0696db276a664e8397d0c3f90e526df8&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=9f09201f0de74baf927082c77fc97d7b => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.677816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01801-fdf2-4596-886c-b285abdac9d9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.678170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01801-fdf2-4596-886c-b285abdac9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.678370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01801-fdf2-4596-886c-b285abdac9d9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.680065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 130/270] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:10.685954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3996-b4c5-4d0b-b131-2916aeb47747 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.686283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3996-b4c5-4d0b-b131-2916aeb47747 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.686797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3996-b4c5-4d0b-b131-2916aeb47747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.714426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e57f3996-b4c5-4d0b-b131-2916aeb47747 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:10.715693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e57f3996-b4c5-4d0b-b131-2916aeb47747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:10.800588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.806699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 141/271] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:10.852290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b9fe-9ca5-48eb-8c16-30d21bb8a4ef None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.852902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b9fe-9ca5-48eb-8c16-30d21bb8a4ef None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.853404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b9fe-9ca5-48eb-8c16-30d21bb8a4ef None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.858634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 131/272] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:10.864109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690570-0af3-46a0-a2ce-b96beddbb61c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.864393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690570-0af3-46a0-a2ce-b96beddbb61c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.864669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690570-0af3-46a0-a2ce-b96beddbb61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.887573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-23690570-0af3-46a0-a2ce-b96beddbb61c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:10.888751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-23690570-0af3-46a0-a2ce-b96beddbb61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:10.973235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 142/273] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:11.110989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b9fe-9ca5-48eb-8c16-30d21bb8a4ef None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.164546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb066a6-a3af-48fc-b67a-b55d4fd61e9c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.165093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb066a6-a3af-48fc-b67a-b55d4fd61e9c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.165519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb066a6-a3af-48fc-b67a-b55d4fd61e9c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.167035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 143/274] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:11.172890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.173174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.173447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.196070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:11.196864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:11.243445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b9fe-9ca5-48eb-8c16-30d21bb8a4ef None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.286055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 144/275] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:11.287321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.287830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.288973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.289394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.289831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.300842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:11.307088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:11.308044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:11.308841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:11.311115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 132/276] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:11.323550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.358575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.403972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.414228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487c996-1f90-4d49-8aaa-98f09a5d49d4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.462163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.463044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c20-e9f9-4e75-abd3-74f84b2ec11e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.463393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.464170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.464836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.467057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.468683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.470465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 145/277] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:11.476323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.503514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.504461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.505012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.506099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.507053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:11.507547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.509174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.519272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 133/278] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:11.525801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.601027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bdef6-8499-4e0b-bfc9-4f4095c04e57 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.629409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.643046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.643666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.643950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.644224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.644892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.646436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.658165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99129793-73bf-4909-944d-afa7e0629cb6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.669109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 146/279] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:11 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:29:11.670319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.692467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.731948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.732619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.732902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.733177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] PATH_INFO: `/v3/users/a43f1441dd04494bb769d694c6103360` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.748789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f1441dd04494bb769d694c6103360)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.750396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a840bb-610a-4829-aa58-a1cc7ae800d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.752198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 134/280] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/users/a43f1441dd04494bb769d694c6103360 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:11.753478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffe09f-9778-43ee-b34b-14e1ef162e21 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.753762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffe09f-9778-43ee-b34b-14e1ef162e21 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.754035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffe09f-9778-43ee-b34b-14e1ef162e21 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.759413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 135/281] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:11.759413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffe09f-9778-43ee-b34b-14e1ef162e21 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.800261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c97a-a073-4516-b17a-43603b6b0ec6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.837856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.838549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.838988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.839358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.854758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2df52e7a-ba4d-40f1-81a8-ec42f8b6d044 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:11.860867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:11.861725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 147/282] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:11.862573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.863302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.863302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.882499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:11.883698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:11.886223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffe09f-9778-43ee-b34b-14e1ef162e21 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.928191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.929166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.929453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.929722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.931626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, user.id=a43f1441dd04494bb769d694c6103360, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:11.931626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.932718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.933597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.940597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 136/283] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/role_assignments?user.id=a43f1441dd04494bb769d694c6103360&scope.system=all&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:11.941274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.983690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 148/284] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:12.006834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.093286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d6e23-edf3-461b-91b1-0f1d0c9a4169 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.134077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.134732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.135007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.135273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.136043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.136236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.137622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.148355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 137/285] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.154131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.270137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badc354-630c-48ad-b13f-11bf2adc0f0d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.294427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.307030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.307935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.308385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.308794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] PATH_INFO: `/v3/users/47fbf635972745fcb0092020e0197319` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.310090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] RBAC: Authorizing `identity:get_user(user_id=47fbf635972745fcb0092020e0197319)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.312091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.314169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 149/286] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/users/47fbf635972745fcb0092020e0197319 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.320687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.386884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.448501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add8c87-f572-4634-b88b-38bfb24002db None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.506271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.506933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.507189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.507460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.538968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51482f5-baec-4988-b6a5-d75a972d8b9a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.540641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.604865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.605506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.605786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.606059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.607406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.608884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.629574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:12.635174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:12.636029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:12.636827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 138/287] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:12.642269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.693807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.753619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 150/288] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:29:12.759699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.862519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021df746-75c5-4ebc-8141-4e3105e63abe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.924411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829419f-53ac-4543-89c9-a25c23a68e7a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.929657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.930278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.930587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.930864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.931534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.931840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.933233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.955013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 139/289] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.961993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.998010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.998676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.998959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.999234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.001620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.003159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.005066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 151/290] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.012243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.013623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.036207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.123131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889cc67-b04e-447f-99ab-5532b1cec4ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.175263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.176228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.176741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.177800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] PATH_INFO: `/v3/users/c4cdbca6890c478e90a2c143db4c8733` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.180591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cdbca6890c478e90a2c143db4c8733)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.182221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.184024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 140/291] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/users/c4cdbca6890c478e90a2c143db4c8733 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:13.189925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.228741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d63e-d5ab-4c02-a27b-00e7036d2ac9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.307356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.308327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.308820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.309280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.310197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=47fbf635972745fcb0092020e0197319) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.310962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.313937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.326562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 152/292] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/role_assignments?user.id=47fbf635972745fcb0092020e0197319&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.379351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.392821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a0403-bc23-4822-b30f-59436b2d4354 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.442490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.443193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.443476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.443753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.451408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.452901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651644bd-f7d8-44de-bc1b-2870c910cae0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.454704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 141/293] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.460600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.565641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.597657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0c60-adba-4597-9454-2fdc1380d2bf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.633200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.634218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.634539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.634924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/c4cdbca6890c478e90a2c143db4c8733/roles/90c95706f60e4a8794b9fb3547e3b0ff` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.637737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, user_id=c4cdbca6890c478e90a2c143db4c8733, role_id=90c95706f60e4a8794b9fb3547e3b0ff)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.639907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993191c-be10-4f10-bc48-31cbd00e3294 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.657377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 153/294] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:13 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/users/c4cdbca6890c478e90a2c143db4c8733/roles/90c95706f60e4a8794b9fb3547e3b0ff => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:14.360921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8c0f-5498-47b0-9837-a9a8d8e047d5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.361583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8c0f-5498-47b0-9837-a9a8d8e047d5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.362051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8c0f-5498-47b0-9837-a9a8d8e047d5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.364114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 142/295] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:14.370852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a9bd-c0fe-40be-bceb-d60b1e95a1fb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.371492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a9bd-c0fe-40be-bceb-d60b1e95a1fb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.371960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a9bd-c0fe-40be-bceb-d60b1e95a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.392392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd27a9bd-c0fe-40be-bceb-d60b1e95a1fb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:14.393925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd27a9bd-c0fe-40be-bceb-d60b1e95a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:14.429027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78522003-dc20-4193-8f16-3f6dc02a8fd7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.430001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78522003-dc20-4193-8f16-3f6dc02a8fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.430503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78522003-dc20-4193-8f16-3f6dc02a8fd7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.432656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 143/296] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:14.438331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.438893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.439378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.459521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:14.461017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:14.477082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 154/297] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:14.492739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658986d-970b-428e-9591-cb25ec2fadd5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.493062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658986d-970b-428e-9591-cb25ec2fadd5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.493301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658986d-970b-428e-9591-cb25ec2fadd5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.494734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 155/298] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:14.499846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.500257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.500632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.518867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:14.519984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:14.538784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 144/299] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:14.585896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.602112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 156/300] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:14.751882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.811993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.848756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.858585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008182e-ec7d-412c-9e87-8779246df30a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.909497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.910200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.910510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.910784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.921820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:14.926723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:14.927682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.928484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 145/301] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:14.929737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.938553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82813-84de-4a01-9e77-905cecd8e2ff None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.975054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.975694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.975970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.976239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.977357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.978847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.984800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 157/302] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:14.984800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.042766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff45a3-81cb-4452-a3d5-7e0fe0718270 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.049110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.078310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.078882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.079157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.079427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.090516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:15.096635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:15.097540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:15.098328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:15.099067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3da1-bb3a-404a-9b9b-59df5e829e58 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.099640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 146/303] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:15.100559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.116752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.134983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.135918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.136205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.136482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.137152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:15.137461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.138874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.147728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 158/304] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:15.148525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.222984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b470-cc1b-4626-9798-0954d50ddb8a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.254662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13610-aa1d-41c3-8e56-e331bd833f0d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.267402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.268125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.268474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.268815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] PATH_INFO: `/v3/groups` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.269522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.271572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.279947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.290056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 147/305] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Sep 28 15:29:15 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:15.290994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.295449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.296741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.297230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.299033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.300196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:15.300752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.302535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.318846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 159/306] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:15.321425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.379531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.417998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49ec4-a69f-4637-a3d1-9340997285ae None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.462205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd0e9-e5bb-4618-85da-8d46c579e2d2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.471844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.473068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.473745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.474085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] PATH_INFO: `/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.490998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] RBAC: Authorizing `identity:get_user(user_id=976a8467d24a4e5fa38c837dbe2b2cfa)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.492790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.495923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 148/307] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:15.500813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.503705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.506084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.506585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.507031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] PATH_INFO: `/v3/users/a43f1441dd04494bb769d694c6103360` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.508405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f1441dd04494bb769d694c6103360)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.510577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.512862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 160/308] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/users/a43f1441dd04494bb769d694c6103360 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:15.519661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.593290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.627181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547802-fd3b-4782-9bc8-28025eeec325 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.649053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d2d88-b554-419d-acc1-d7b7bdf684d2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.664648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.675525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.676544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.677086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.677808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.679383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, user.id=976a8467d24a4e5fa38c837dbe2b2cfa, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:15.680004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.681834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ed2b-4d49-4bee-9e0f-33d6f9e9b9d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.686914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.687701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.688042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.688455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] PATH_INFO: `/v3/system/users/a43f1441dd04494bb769d694c6103360/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.688910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 149/309] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/role_assignments?user.id=976a8467d24a4e5fa38c837dbe2b2cfa&scope.system=all&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:15.691943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a43f1441dd04494bb769d694c6103360, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.694058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3be1d-ba5c-4454-9289-eff166a3c175 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.714662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 161/310] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Sep 28 15:29:15 2022] PUT /identity/v3/system/users/a43f1441dd04494bb769d694c6103360/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:29:15.956452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992705-9823-407b-9eb4-a35f68288caa None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.956689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992705-9823-407b-9eb4-a35f68288caa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.956926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992705-9823-407b-9eb4-a35f68288caa None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.958830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 150/311] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:15.964490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.964779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.965055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.983813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:15.985337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:16.058208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 162/312] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:16.325979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc992705-9823-407b-9eb4-a35f68288caa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.439081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc992705-9823-407b-9eb4-a35f68288caa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.475312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.475968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.476221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.476498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.487526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:16.493667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:16.494595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:16.495402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 151/313] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:16.502835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.549968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.607591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc047-880a-477f-8179-d81cfd715cdf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.642242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.642890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.643171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.643446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.644115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:16.644417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:16.645840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:16.655112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 163/314] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:16.660960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.721300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.770866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6bf4-ee7a-4c88-a882-667a67ab52ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.809450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.810014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.810290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.810591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] PATH_INFO: `/v3/users/c4cdbca6890c478e90a2c143db4c8733` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.811954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cdbca6890c478e90a2c143db4c8733)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:16.813393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:16.815004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 152/315] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/users/c4cdbca6890c478e90a2c143db4c8733 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:16.821443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.929200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495be7-2cf2-400b-a0c8-d4146de62a25 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.943088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.963467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b907c-3553-493b-9829-93536d5be540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.964072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b907c-3553-493b-9829-93536d5be540 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.964347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b907c-3553-493b-9829-93536d5be540 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.964616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b907c-3553-493b-9829-93536d5be540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.034076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b907c-3553-493b-9829-93536d5be540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.035568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b907c-3553-493b-9829-93536d5be540 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.149668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 164/316] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:17.155258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.263022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464ff1c-f84f-42bf-bbf6-116137cfe900 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.283562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.298561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.299246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.299530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.299804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.300962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.304949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c0452-84dc-4f3b-94cd-828ada921601 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.304949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 153/317] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:17.310608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b907c-3553-493b-9829-93536d5be540 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.321729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.420988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b907c-3553-493b-9829-93536d5be540 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.458861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.459791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.460274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.460768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.462159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5, user.id=c4cdbca6890c478e90a2c143db4c8733) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:17.462747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.465605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262735-eafd-426f-8f0b-b2701677efe5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.473278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 165/318] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/role_assignments?user.id=c4cdbca6890c478e90a2c143db4c8733&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:17.591850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687584d-24d3-4879-b979-e49070572001 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.592791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687584d-24d3-4879-b979-e49070572001 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.593246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687584d-24d3-4879-b979-e49070572001 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.595115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 154/319] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:17.601498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5787-b694-4651-a9b0-e17cabd66057 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.602147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5787-b694-4651-a9b0-e17cabd66057 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.602649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5787-b694-4651-a9b0-e17cabd66057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.622802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-645a5787-b694-4651-a9b0-e17cabd66057 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:17.624837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-645a5787-b694-4651-a9b0-e17cabd66057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:17.705049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 166/320] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:17.971048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687584d-24d3-4879-b979-e49070572001 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.047752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a7134-0330-4375-b6d9-671c7ae8e92e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.048030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a7134-0330-4375-b6d9-671c7ae8e92e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.048304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a7134-0330-4375-b6d9-671c7ae8e92e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.049643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 167/321] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:18.050678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1120f0-05bf-4364-8243-69d2836d5c9d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.050956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1120f0-05bf-4364-8243-69d2836d5c9d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.051226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1120f0-05bf-4364-8243-69d2836d5c9d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.052748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 168/322] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:18.056928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.057208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.057521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.067453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.079737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687584d-24d3-4879-b979-e49070572001 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.080119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:18.081307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:18.114595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.115229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.115507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.115781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.116867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.118303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7c164-964d-410d-93a0-1add92003032 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.119949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 155/323] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:18.121147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.121432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.121702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.139461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:18.140655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:18.149293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 169/324] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:18.150114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.194205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.214457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 156/325] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:18.266393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181b2a-f675-44d6-a93f-709e5ae3036e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.297405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.303915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.304806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.305416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.305689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] PATH_INFO: `/v3/groups` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.306608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.308337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.325280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 170/326] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Sep 28 15:29:18 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:18.418283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.499345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.525900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f7672-c614-490f-b3ba-730377dfc72e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.566097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.567115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.567403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.567720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.582148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:18.587987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:18.588949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:18.589739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 157/327] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:18.595152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.627601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d235-1d79-4e65-b0b1-f436ec2fc46c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.658400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.666349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.667014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.667293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.667558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.681378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:18.686992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:18.687888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:18.688871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 171/328] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:18.693816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.730045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db07720-35a4-45f8-8e0a-94230f4bd3c1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.760872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.771485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.772309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.772442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.772938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.773802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.773802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.775021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.785091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 158/329] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:18.792647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.828295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d671-8beb-43fe-b089-9ceb7bb4e309 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.866779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.867415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.867711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.867986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.868657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.868963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.870387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.881271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 172/330] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:18.889506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.946731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d679db-4773-4627-95c1-48889fe81655 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.959716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.995603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.996270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.996554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.997385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] PATH_INFO: `/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.998890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] RBAC: Authorizing `identity:get_user(user_id=976a8467d24a4e5fa38c837dbe2b2cfa)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.000446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.002215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 159/331] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:19.007853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.015309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212682c6-0c81-40a2-a388-5fc8aae9b63c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.034889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.063074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.063883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.064644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.064929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] PATH_INFO: `/v3/users/a43f1441dd04494bb769d694c6103360` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.066125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f1441dd04494bb769d694c6103360)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.068337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.070110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 173/332] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/users/a43f1441dd04494bb769d694c6103360 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:19.076040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.134440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22062bec-9e78-48a6-b0df-f37856a05d2d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.170491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.171159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.171442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.171718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] PATH_INFO: `/v3/system/users/976a8467d24a4e5fa38c837dbe2b2cfa/roles/90c95706f60e4a8794b9fb3547e3b0ff` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.173282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=976a8467d24a4e5fa38c837dbe2b2cfa, role_id=90c95706f60e4a8794b9fb3547e3b0ff)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.174762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd2e3f-e438-402a-ac87-9a1e3d133505 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.187702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 160/333] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Sep 28 15:29:19 2022] PUT /identity/v3/system/users/976a8467d24a4e5fa38c837dbe2b2cfa/roles/90c95706f60e4a8794b9fb3547e3b0ff => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:19.218275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f347-5e79-4ad0-afe7-9b17114865e9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.259602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.260239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.260523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.260795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.263469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] RBAC: Adding query filter params (role.id=5a6068f0cdfb4778a8837b31039dbeb7, user.id=a43f1441dd04494bb769d694c6103360, scope.system=all) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:19.264014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.265587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c521d0-9e78-4508-bc9f-f20853f4b675 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.286860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 174/334] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/role_assignments?user.id=a43f1441dd04494bb769d694c6103360&scope.system=all&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:20.947466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02384-e439-4d1a-af60-14b5d515a04f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.948008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02384-e439-4d1a-af60-14b5d515a04f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.948008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02384-e439-4d1a-af60-14b5d515a04f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.949579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 161/335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:20.972243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62b67-228c-4071-8ded-3bdefab99189 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.973156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62b67-228c-4071-8ded-3bdefab99189 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.974197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62b67-228c-4071-8ded-3bdefab99189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.003558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-40a62b67-228c-4071-8ded-3bdefab99189 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:21.005250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-40a62b67-228c-4071-8ded-3bdefab99189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:21.044628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:21.084289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 175/336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:21.526102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6af7-40a3-454f-b812-d9e11b993115 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.526470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6af7-40a3-454f-b812-d9e11b993115 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.526646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6af7-40a3-454f-b812-d9e11b993115 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.528287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 162/337] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:21.533786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.534074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.534353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.563441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:21.564651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:21.651367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 176/338] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:21.704418 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6af7-40a3-454f-b812-d9e11b993115 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:21.769217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:21.847220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6af7-40a3-454f-b812-d9e11b993115 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:21.887960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:21.888708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.888989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.889273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.904209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:21.909609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:21.910552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:21.911390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 163/339] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:21.916986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.067783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.071020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9a71-787a-4e34-8a68-aa86c73e8c88 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.121036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.121567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.121838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.122106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.122806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:22.123107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.124487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fa015-e1b9-4610-802a-23692c239851 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.138518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 177/340] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:22.143937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.159370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6d50-7b90-4e98-beb6-13d41ccd5a39 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.159679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6d50-7b90-4e98-beb6-13d41ccd5a39 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.159925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6d50-7b90-4e98-beb6-13d41ccd5a39 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.161446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 178/341] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:22.167550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd54e-f7f5-4e35-a4b3-119d50de2e88 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.167846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd54e-f7f5-4e35-a4b3-119d50de2e88 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.168098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd54e-f7f5-4e35-a4b3-119d50de2e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.199500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fd5fd54e-f7f5-4e35-a4b3-119d50de2e88 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:22.200714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fd5fd54e-f7f5-4e35-a4b3-119d50de2e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:22.282247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.285914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffdfe1-763c-4e54-811f-e3f8da56b310 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.327702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 179/342] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:22.329634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4de95-e7e6-4540-af86-45ebae4a2e55 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.329743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.330270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4de95-e7e6-4540-af86-45ebae4a2e55 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.330373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.330763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.331041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.331240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4de95-e7e6-4540-af86-45ebae4a2e55 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.338697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 180/343] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:22.341152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4de95-e7e6-4540-af86-45ebae4a2e55 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.467804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.469638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b1068-c165-48b1-b1c7-1fb4cae98571 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.523726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.551125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4de95-e7e6-4540-af86-45ebae4a2e55 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.584159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.587101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 164/344] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:22.592215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.592879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.593351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.624918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:22.627889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:22.634669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.634986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.635175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.635446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.661755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:22.666801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:22.667722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:22.668523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.671354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 181/345] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:22.676220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.724755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 165/346] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:22.725908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.793246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0627d-74f8-49c2-aa3e-9a916041e113 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.832655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6a417-d074-4ac7-9143-6515846d99c4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.834366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.835052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.835337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.835753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.837499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.838397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.840107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 182/347] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:22.840988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.868312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.871472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.877493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.878404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.878921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.879417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.881007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:22.881507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.883517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.892577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 166/348] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:22.893818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.965005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a50c1-0ddd-4575-a135-4e9fded48948 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.004810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.005467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.005754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.006029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.013216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf9b3-f1d2-4b61-aeb4-5a00534f73cd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.020529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:23.027411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:23.028350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:23.029171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 183/349] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:23.029914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.053701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.054748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.055230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.055737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.062777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.064681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.066712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 167/350] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.067814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.134391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.175463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c61c49-3542-457f-9df1-1cbca4ea5454 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.190920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.216916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.217931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.219608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.220200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.221118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:23.221625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.223289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.228176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32fbf0-957b-4982-912e-899a8ea0c69c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.234095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 168/351] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.235361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.284889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.285587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.285981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.286772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.368095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2848465-c0ed-4f20-b147-f8c776bd7772 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.381891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.383553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.392735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.423115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.423728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.424007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.424277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.425010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:23.425316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.426840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.433017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 169/352] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.433781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.505223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.536946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 184/353] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:23.537742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.581836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657fe8f-19cf-4cf8-927e-4d7b19605de6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.628229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.629268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.630289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.630812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.644540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-17b4c5ea-3e70-443d-ae87-75c8a045f1dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:23.654891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:23.655929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:23.656763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 170/354] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:23.657539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe9cf-1610-4994-9c8e-499513e2e4e4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.657821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe9cf-1610-4994-9c8e-499513e2e4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.658102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe9cf-1610-4994-9c8e-499513e2e4e4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.659754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 171/355] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:23.660315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebe9cf-1610-4994-9c8e-499513e2e4e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.698680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac398ff-2d83-4386-8e72-cdb71e2d53c8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.734323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.741569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.742210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.742525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.742877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] PATH_INFO: `/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.743971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] RBAC: Authorizing `identity:get_user(user_id=976a8467d24a4e5fa38c837dbe2b2cfa)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.745415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.747836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 185/356] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/users/976a8467d24a4e5fa38c837dbe2b2cfa => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.748625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.812158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebe9cf-1610-4994-9c8e-499513e2e4e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.815889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.855841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.856847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.857403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.857885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.859382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.861034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de748d39-9b2b-4503-b687-0bda96da3da8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.862947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 172/357] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.864853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b01d8-6fad-496e-8d17-9f0d640e11ed None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.865289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b01d8-6fad-496e-8d17-9f0d640e11ed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.865722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b01d8-6fad-496e-8d17-9f0d640e11ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.879991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad8d8-6a7f-4afb-b635-6356b4a0767e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.884304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-630b01d8-6fad-496e-8d17-9f0d640e11ed None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:23.885633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-630b01d8-6fad-496e-8d17-9f0d640e11ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:23.919458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.920259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.920621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.921046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.921835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:23.922144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.923584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.933643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 186/358] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.934447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.972632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 173/359] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:23.975194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901665e-20af-4da4-af4d-58124189e375 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.975491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901665e-20af-4da4-af4d-58124189e375 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.975755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901665e-20af-4da4-af4d-58124189e375 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.977151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 174/360] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:23.978001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901665e-20af-4da4-af4d-58124189e375 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.033689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.062460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7837d5a-8532-4aa5-8e54-6489cedb2c61 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.075960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.106141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901665e-20af-4da4-af4d-58124189e375 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.108355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.110064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.110537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.110958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.111815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] RBAC: Adding query filter params (role.id=90c95706f60e4a8794b9fb3547e3b0ff, user.id=976a8467d24a4e5fa38c837dbe2b2cfa, scope.system=all) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.112274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.114746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.134374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 187/361] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/role_assignments?user.id=976a8467d24a4e5fa38c837dbe2b2cfa&scope.system=all&role.id=90c95706f60e4a8794b9fb3547e3b0ff => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:24.136101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.147176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.148004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.148373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.148746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.150274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.151939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7cb30-3c96-47fc-b53e-0c776f92a6f9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.153855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 175/362] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:24.156365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.157033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.157408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.183960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:24.185439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:24.306106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116f3b-9b8b-4769-a208-e964e068cbb3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.312100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 176/363] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:24.314474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.369716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.370734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.371176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.371606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.398325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.437809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.495822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.497324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.503252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e39f0-53ef-4bea-b822-288718aa4c93 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.550884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.552377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.552800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.553236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.554251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.554737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.556661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.564699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 177/364] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=9f09201f0de74baf927082c77fc97d7b => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:24.565971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.661480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 188/365] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:24.666899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.711205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082ef8-1ca5-4a0d-a786-ce1304e0f681 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.731495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.758473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.759208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.759536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.759806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.773359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:24.775082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:24.780277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.781293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.782102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 178/366] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:24.782964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.824045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f892846-abb0-4b9b-bf73-5533de7cb4ee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.879407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.881146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.881803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.882363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.884009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.886618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.888814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 189/367] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:24.889868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.929564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f9f4-f9ae-40a7-939b-a87629655255 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.975157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.975849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.976130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.976453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.990050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:24.995727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.996639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 179/368] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:24.997837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.026975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.032917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965baef-a0bf-4b6c-a066-b87fb3155f23 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.075360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.076539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.077576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.078054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.078942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:25.079598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.081330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.085659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.095544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 190/369] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.097494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.120627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237ba65-afe5-44d7-b39b-77f74c30a159 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.166315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.168212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.168700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.169193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.170992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.173190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.175291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 180/370] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.176453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.240816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbba1a-325b-4481-a4ec-98189112561c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.292596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.293212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.293494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.293788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.294461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:25.294779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.296177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.307328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 191/371] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.314195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.317661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.332753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2975c-941d-40df-b38e-b6ac5b784515 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.380831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.381845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.382317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.382797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.387521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.451730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717a6f8-58a7-4211-96ea-9178aa96bcc4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.479045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.481265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.492050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.492968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.493412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.493821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.495704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:25.496158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.497758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.507236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 192/372] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.508107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.619497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.638497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 181/373] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:25.640805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.652217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181f41-5277-493d-9726-0a33b26c25e5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.684724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.697904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.698947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.699613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.699951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.775196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277c83-31b3-4957-9475-0937e083bb26 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.794391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.796233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.812638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.813378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.813725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.814088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.815352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.816865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.823723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 182/374] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:25.825458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.890661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.914206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 193/375] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:25.923474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.939314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86d8ed-f130-4d3e-b053-d37d84231ca4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.960298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.010920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.011843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.012269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.012684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.014193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.017142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.022777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 183/376] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:26.027916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.089169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047237ec-d949-4854-8a00-2a48fc206995 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.163383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.164032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.164310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.164582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] PATH_INFO: `/v3/projects/4c663536db924b2aba9e9bb0e3254711` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.165792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] RBAC: Authorizing `identity:get_project(project_id=4c663536db924b2aba9e9bb0e3254711)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.167335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.169351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 194/377] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/projects/4c663536db924b2aba9e9bb0e3254711 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.179453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.223900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66db90-8edb-4059-9e6f-6c4fc5ad53fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.263715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.276186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.276793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.277078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.277351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.278193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.278534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.279970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.287648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 184/378] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=9f09201f0de74baf927082c77fc97d7b => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.354843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.372853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1043d-59ae-4b75-889d-3039710d1042 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.425014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.425581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.425863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.426137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] PATH_INFO: `/v3/groups/72c505f99d0445898757142985445055` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.437077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] RBAC: Authorizing `identity:get_group(group_id=72c505f99d0445898757142985445055)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.438598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da77d9-955a-447c-a342-2a9991320df1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.440641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 195/379] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/groups/72c505f99d0445898757142985445055 => generated 229 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:26.446533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.492028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e882-7c2e-41ed-80eb-c0e09698ac81 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.492347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e882-7c2e-41ed-80eb-c0e09698ac81 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.492585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e882-7c2e-41ed-80eb-c0e09698ac81 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.494472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 196/380] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:26.500204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.500603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.500883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.521204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:26.522401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:26.612618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eaa0c-dd49-4970-b739-91e964162a43 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.660617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 197/381] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:26.669157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.670086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.670599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.671076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.671989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] RBAC: Adding query filter params (group.id=72c505f99d0445898757142985445055, role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=4c663536db924b2aba9e9bb0e3254711) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.672506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.674139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.681304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 185/382] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/role_assignments?group.id=72c505f99d0445898757142985445055&scope.project.id=4c663536db924b2aba9e9bb0e3254711&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.954859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.057089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.080494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3727db-8ca9-4239-8527-99218780e030 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.135172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.135811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.136086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.136357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.147845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:27.153068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:27.154037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:27.155222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 198/383] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:27.163197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.268265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.327418 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e963ab-590c-4f71-b754-95502efd5b4d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.381406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.382020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.382295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.382591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.383257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:27.383555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.384936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5ade-6ae3-4717-a824-68f69080a499 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.398864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 186/384] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:27.404500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.566731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936e978-8427-47f3-8adf-6201e6ccdc84 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.610925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.611568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.611849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.611955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808869-3256-497d-99c4-cc595eef75bb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.612378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.612451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808869-3256-497d-99c4-cc595eef75bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.612451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808869-3256-497d-99c4-cc595eef75bb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.613355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.614402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 187/385] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:27.615545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c184946-026d-4831-9554-c19c4fb0aa1e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.617447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 199/386] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:27.623772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed808869-3256-497d-99c4-cc595eef75bb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.624175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.624467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.624742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.658650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:27.660270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:27.690235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.800744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed808869-3256-497d-99c4-cc595eef75bb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.828117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 200/387] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:27.828717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.845046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.845697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.845980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.846310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.847523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.849623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.851746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 188/388] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:27.858043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.010700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca33e3-de14-473b-9afc-873ab87e470a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.047876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.048547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.048826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.049103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/groups/9c3016b63ba14b7db0418b3889fdc653/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.051019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, group_id=9c3016b63ba14b7db0418b3889fdc653, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.052518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.073106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 201/389] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 15:29:27 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/groups/9c3016b63ba14b7db0418b3889fdc653/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:28.181927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.298402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac24c8-57ab-428c-97ca-63c355544d4c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.342541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.343203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.343482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.343751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.361172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b527abcd-ca84-4e6c-a89a-27d821ec1418 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:28.367253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:28.368219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:28.369644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 189/390] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:28.374656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.378771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.467542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33512-b31a-427d-807f-0c568f62c55e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.467838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33512-b31a-427d-807f-0c568f62c55e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.468006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33512-b31a-427d-807f-0c568f62c55e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.469647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 190/391] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:28.482324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.482663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.482914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.506905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bb83-a648-488e-bce5-ca7fcac913ce None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.514200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:28.516818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:28.550361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.551186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.551398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.551643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.552367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:28.552678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.554171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.563958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 202/392] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:28.569871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.573376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.613132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 191/393] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:28.661882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.708165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0de253-b9a6-4857-a6d1-0041ef65bd27 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.744432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.745400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.745877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.746659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.747992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.749707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.751918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 203/394] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:28.757207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.870476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25163a91-e9af-4cd0-9c63-cc9d26113518 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.911240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.911875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.912178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.912448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.913578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.915081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.916730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 192/395] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:28.925885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.075020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f77-5397-43f4-8fd5-cf9dccad1476 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.121682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.122344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.122645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.122919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/groups/9c3016b63ba14b7db0418b3889fdc653/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.128180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb90ef4cfe4229b292dd5884e54a6b, group_id=9c3016b63ba14b7db0418b3889fdc653, role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.130162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934272-06c7-425d-bef0-4c439fbad96a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.132505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.144153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.151382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 204/396] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 15:29:28 2022] PUT /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b/groups/9c3016b63ba14b7db0418b3889fdc653/roles/9f09201f0de74baf927082c77fc97d7b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:29.216063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.308286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed110b-739a-4f37-a528-c5317d7935d2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.361093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.361670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.361949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.362224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.383494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:29.388552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:29.389462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:29.390731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 193/397] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:29.396292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db8ec-9d59-4255-8af4-562448231cb5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.396593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db8ec-9d59-4255-8af4-562448231cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.396870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db8ec-9d59-4255-8af4-562448231cb5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.397492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.402082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 205/398] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:29.404313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.404598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.404883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.425438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:29.427543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:29.534224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d4245-c18e-4c2a-866b-da9bc6816ca3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.536393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 206/399] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:29.536994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.573920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.574594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.574877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.575149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.575830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:29.576134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.577561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe7c2-10b6-416e-9752-26cb4d28b0f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.587758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 194/400] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:29.593654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.712311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d05f-d837-493f-af11-875a86142e27 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.712961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d05f-d837-493f-af11-875a86142e27 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.712961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d05f-d837-493f-af11-875a86142e27 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.714496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 195/401] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:29.719448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.719732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.720026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.752045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654010-6737-45f5-b0f0-dc5a66044760 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.763136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:29.764313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:29.793378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.805691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.806685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.807175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.807700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.809107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.810755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.813384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 207/402] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:29.819635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.905439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 196/403] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:29.919514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.967020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca6f9-9fef-4c3d-b710-ce938bc41397 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.006266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.006923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.007346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.007630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.008908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.010679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.012674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 208/404] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:30.018172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.111078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.130223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.143559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ca4dd-32cd-4c30-9ac8-a8041bab5ff4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.182245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f7269-3208-4b92-8f97-fd411248bded None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.210016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.211027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.211480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.212034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.220657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.221348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.221689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.222064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/groups/9c3016b63ba14b7db0418b3889fdc653/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.225177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, group_id=9c3016b63ba14b7db0418b3889fdc653, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.226998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.234245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:30.239214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:30.240081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:30.240896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:30.243540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 209/405] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 15:29:30 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/groups/9c3016b63ba14b7db0418b3889fdc653/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:30.244465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.245265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 197/406] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:30.246146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.456527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888204-a007-480a-bd35-c92359a987cc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.476582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64f22e-4a65-4a72-a54e-28f1e857978b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.482775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.498298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.498979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.499267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.499544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.511722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:30.517510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:30.518439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:30.519245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 210/407] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:30.525156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.530619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.531254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.531527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.531811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.532482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.532786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.535230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.553094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 198/408] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:30.559195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.601909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.745372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb856-e211-47d8-90d9-8089c3c554c2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.781872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f7a9d-b457-4a1a-86e1-7009a7116a58 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.816356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.817518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.817803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.818075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.818776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.819093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.820734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.832635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 211/409] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:30.841663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.869003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.869638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.869913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.870182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.871473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.873037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48599e0a-fd00-413b-87b9-4c7880ad8b70 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.875094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 199/410] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:30.876287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d263-c055-4381-85a4-6eb5802db082 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.876567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d263-c055-4381-85a4-6eb5802db082 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.876838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d263-c055-4381-85a4-6eb5802db082 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.878131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 200/411] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:30.884222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d263-c055-4381-85a4-6eb5802db082 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.902834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.968299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8666a45-f565-4809-bccf-3c4449529bef None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.001773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.005103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.006008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.006555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.007037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] PATH_INFO: `/v3/groups/72c505f99d0445898757142985445055` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.007139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d263-c055-4381-85a4-6eb5802db082 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.008467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] RBAC: Authorizing `identity:get_group(group_id=72c505f99d0445898757142985445055)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.010118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf377a-6cf2-43f7-8cfa-c6595b8a336f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.012537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 212/412] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/groups/72c505f99d0445898757142985445055 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:31.013874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.014392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.014891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.044079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:31.045557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:31.053856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.054884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.055409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.056546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.058364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.060687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.063523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 201/413] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:31.064577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.145412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 213/414] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:31.145412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.215018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ec72-b74f-4a91-808f-d440f6f1a882 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.224941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.255080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.255645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.255922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.256193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] PATH_INFO: `/v3/projects/4c663536db924b2aba9e9bb0e3254711` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.257327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] RBAC: Authorizing `identity:get_project(project_id=4c663536db924b2aba9e9bb0e3254711)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.260688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.260688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 202/415] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/projects/4c663536db924b2aba9e9bb0e3254711 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:31.265993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.309713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.352691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e0737-f7e1-4a8b-bc36-b44642943386 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.413593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.414655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.415157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.415620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/groups/9c3016b63ba14b7db0418b3889fdc653/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.417931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5, group_id=9c3016b63ba14b7db0418b3889fdc653, role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.420486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.445573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 214/416] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 15:29:31 2022] PUT /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5/groups/9c3016b63ba14b7db0418b3889fdc653/roles/9f09201f0de74baf927082c77fc97d7b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:31.450032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2970a-0b95-4396-b9e0-c581e95667e1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.488573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.490288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.491111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.491616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] PATH_INFO: `/v3/projects/4c663536db924b2aba9e9bb0e3254711/groups/72c505f99d0445898757142985445055/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.493880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c663536db924b2aba9e9bb0e3254711, group_id=72c505f99d0445898757142985445055, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.495602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e026e6-d3d3-45ed-a1d7-9cf8e3635b17 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.519637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 203/417] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 15:29:31 2022] PUT /identity/v3/projects/4c663536db924b2aba9e9bb0e3254711/groups/72c505f99d0445898757142985445055/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:29:31.550282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.678383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.683030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a0a4-234b-43b1-80d8-7fc06c03cae0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.693988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77fae-a91f-4058-bc84-20d74599dc5d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.693988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77fae-a91f-4058-bc84-20d74599dc5d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.693988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77fae-a91f-4058-bc84-20d74599dc5d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.694668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 204/418] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:31.702244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.702668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.703022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.726371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:31.727900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:31.731420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.732062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.732343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.732617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.744832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:31.750129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:31.751561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:31.752356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 215/419] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:31.757607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.811908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 205/420] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:31.885905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9016fe-8499-4e0b-80f5-0429f74b10a5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.923400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.924053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.924328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.924594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.925305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:31.925607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.927075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.936205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 216/421] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:31.942217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.955631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.079693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1719d0-a4f8-445f-9eee-6422afc7b89e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.131150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.131771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.132108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.132380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.215308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.216778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.221852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.235135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.277725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.364725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 206/422] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:32.371456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.483123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435ff1f-5ef1-4e60-9613-3e9445a42657 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.529458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc450e-ff4f-4a3b-9210-ac530bdcfd4d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.547632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.552762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.553668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.554167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.554680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.566897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:32.573368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:32.574249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:32.575080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:32.577643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 217/423] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:32.587484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.624773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.625352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.625634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.625911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.631268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.632749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.634814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 207/424] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:32.640468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.711626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3642fbb-6dc2-4e8d-bd5a-2b9f70eaaa12 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.714985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.777794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.779512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.780030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.780519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.781390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:32.782331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.784482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c0480-3414-4fe4-b232-3708f1b9dce9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.796092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 218/425] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:32.797869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8f45-ac25-430b-bd91-1452521f71e4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.798516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8f45-ac25-430b-bd91-1452521f71e4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.799035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8f45-ac25-430b-bd91-1452521f71e4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.801252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 219/426] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:32.802184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8f45-ac25-430b-bd91-1452521f71e4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.837706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1653-c7d5-4a2f-9ff6-a599e28636ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.875681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.876102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.876381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.876659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.877866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.879422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41e344-5e4f-4d6b-ae39-bccc9fd458c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.881094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 208/427] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:32.882203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063539a-53bf-4643-a86a-8c549264952f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.887652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063539a-53bf-4643-a86a-8c549264952f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.887969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063539a-53bf-4643-a86a-8c549264952f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.906746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6063539a-53bf-4643-a86a-8c549264952f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:32.907941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6063539a-53bf-4643-a86a-8c549264952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:32.940909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8f45-ac25-430b-bd91-1452521f71e4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.958728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.985107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.986066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.986624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.987096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.998874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 209/428] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:33.000170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063539a-53bf-4643-a86a-8c549264952f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.004660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:33.064679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.066637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.176636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063539a-53bf-4643-a86a-8c549264952f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.240069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 220/429] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:33.243859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.244737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.246071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.247902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.248412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.249354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:33.250309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.252168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.267129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 210/430] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:33.308959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:33.381514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc834e20-371d-4baf-9a1b-6e5330a9aa89 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.382883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.457840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:33.461867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.463180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.463180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.463180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] PATH_INFO: `/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.464535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] RBAC: Authorizing `identity:get_project(project_id=e9eb90ef4cfe4229b292dd5884e54a6b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.466109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.470780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 221/431] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/projects/e9eb90ef4cfe4229b292dd5884e54a6b => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:33.473799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.567672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcdee-2f35-41b6-a5d3-32ae0dec2e8e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.613228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.613916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.614196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.614485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.626219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293621-054f-42b2-93bb-4a338492c149 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.628662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:33.635997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:33.636977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:33.637772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 211/432] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:33.644143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.648223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:33.672194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.673109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.673601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.674052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.675751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.677584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.679496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 222/433] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:33.686824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.765409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:33.769710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b83c1-f82d-458c-8dd3-e7ef52ad396d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.812896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.813534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.813811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.814082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.814842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:33.815088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.817173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.827333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9fe9-5ac6-41a7-941d-3ab9848ef6e9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:33.828686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 212/434] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:33.835290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:33.872161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:33.873148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.873658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.874269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.875332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:33.875893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:33.877635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36be5f9-edfd-4c9b-a930-23c07fea997f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:33.886199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 223/435] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=e9eb90ef4cfe4229b292dd5884e54a6b&role.id=9f09201f0de74baf927082c77fc97d7b => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:33.963385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52fd9-1f1d-4f72-9d78-14f782fee730 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.003640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.004820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.005099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.005442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.040964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.080393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93094-6f48-4f06-9f2f-42d0b8ed0b53 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.081029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93094-6f48-4f06-9f2f-42d0b8ed0b53 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.081510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93094-6f48-4f06-9f2f-42d0b8ed0b53 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.083550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 224/436] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:34.089547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842d6d7-6dff-4d08-871f-86c3d62401e0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.089722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.090284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842d6d7-6dff-4d08-871f-86c3d62401e0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.090824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842d6d7-6dff-4d08-871f-86c3d62401e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.091221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.116107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3842d6d7-6dff-4d08-871f-86c3d62401e0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:34.117453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3842d6d7-6dff-4d08-871f-86c3d62401e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:34.184467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.219529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 225/437] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:34.221164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176b066-a58b-41d5-a441-c1ed0c1e10cb None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.221455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176b066-a58b-41d5-a441-c1ed0c1e10cb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.221733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176b066-a58b-41d5-a441-c1ed0c1e10cb None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.223822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 226/438] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:34.224318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 213/439] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:34.229165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.229456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.229737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.230254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.256827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:34.260182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:34.315848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.348485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 227/440] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:34.365808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb918-4383-4893-b322-bb9c73a74215 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.403954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.405103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.405609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.406076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.407984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.410328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.413004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 214/441] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:34.418428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.501456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.533068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5ca1-c475-454c-a3fd-13e266772644 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.536340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.575704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.576254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.576537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.576812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.578009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.579462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.581208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 228/442] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:34.586702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.625294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5dfdf-3745-4d9c-bc96-a10c5f452fe3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.654426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.660507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.661141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.661418 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.661698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.671934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:34.677290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:34.678452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:34.679300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 215/443] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:34.679975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.692440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0967c8-1a8b-48cc-a64a-06c31631180a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.729972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.730592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.730878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.731145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.731867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=5a6068f0cdfb4778a8837b31039dbeb7, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:34.732174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.733603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.739941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 229/444] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=5a6068f0cdfb4778a8837b31039dbeb7 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:34.740731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.783627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.788165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429451e1-82f9-4ab4-aa4e-994d8817ba9d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.829440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.830274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.830383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.830693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.843292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:34.848946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:34.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:34.850877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 216/445] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:34.857791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.861364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c6a81-076e-4c85-842c-57cfd28f6c1d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.900456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.901082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.901363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.901635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.902311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:34.902745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.904136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.917672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 230/446] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:34.923503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.924967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.989228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef481fbe-11f9-4511-807a-fa6fdeccbddf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.031683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.032658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.033140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.033624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.034300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.034737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.036674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.045714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 217/447] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.054546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.056002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.059702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf57a-6af8-4646-805a-dfe78ba4848e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.111888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.112702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.112989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.113894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.184147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735922c-cbf5-4f0f-bfe8-eaa0b6009634 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.192867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.194335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.197314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.218815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.219415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.219692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.219958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.290533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.291984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.292530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.297603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 231/448] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:29:35.303248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.393407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 218/449] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:29:35.399396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.399656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67432ee-4d1a-4a5c-a651-db5d616bacc7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.413332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.434118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.435194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.435911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.436387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] PATH_INFO: `/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.438371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb5cfb16d1d4a3890638c2cb8af7da5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.441031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.448485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 232/450] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/projects/cbb5cfb16d1d4a3890638c2cb8af7da5 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.449489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.499958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cd92-f51d-425a-a083-c7e78f5f6ba5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.512929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.533103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.533732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.534010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.534281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] PATH_INFO: `/v3/projects/4c663536db924b2aba9e9bb0e3254711` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.535964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] RBAC: Authorizing `identity:get_project(project_id=4c663536db924b2aba9e9bb0e3254711)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.537398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.539234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 219/451] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/projects/4c663536db924b2aba9e9bb0e3254711 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.544412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.549355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d748d0-2c94-484c-b3f3-fd077d307781 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.581761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.582921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.583208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.583483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] PATH_INFO: `/v3/groups/9c3016b63ba14b7db0418b3889fdc653` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.584595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] RBAC: Authorizing `identity:get_group(group_id=9c3016b63ba14b7db0418b3889fdc653)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.586027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.587621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 233/452] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/groups/9c3016b63ba14b7db0418b3889fdc653 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.593398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.622908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.642259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe2d7-f628-4392-bd2f-e6abd6c548fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.675500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.676118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.676565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.676673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] PATH_INFO: `/v3/groups/72c505f99d0445898757142985445055` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.677745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] RBAC: Authorizing `identity:get_group(group_id=72c505f99d0445898757142985445055)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.679214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.680802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 220/453] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/groups/72c505f99d0445898757142985445055 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.686783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.690660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982703e-a22b-4da2-a76c-0278523b5eae None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.723811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.724408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.724686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.724959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.725686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] RBAC: Adding query filter params (group.id=9c3016b63ba14b7db0418b3889fdc653, role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.725996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.727472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b90e-d028-4b89-a5ba-8746de269829 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.729204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.738795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 234/454] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/role_assignments?group.id=9c3016b63ba14b7db0418b3889fdc653&scope.project.id=cbb5cfb16d1d4a3890638c2cb8af7da5&role.id=9f09201f0de74baf927082c77fc97d7b => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.787404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f5d-a6c2-477b-9fde-5fe15008a6d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.823640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.824262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.824542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.824810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.825559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] RBAC: Adding query filter params (group.id=72c505f99d0445898757142985445055, role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=4c663536db924b2aba9e9bb0e3254711) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.825865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.827341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ba99-9897-46f7-acc8-4104ccf6384a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.834227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 221/455] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/role_assignments?group.id=72c505f99d0445898757142985445055&scope.project.id=4c663536db924b2aba9e9bb0e3254711&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:38.692168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa19fc-0da1-4ca9-b0c8-db353987c376 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.692716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8018f-3ab8-42af-8348-4884d8539b81 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.692716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8018f-3ab8-42af-8348-4884d8539b81 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.693102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa19fc-0da1-4ca9-b0c8-db353987c376 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.693171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8018f-3ab8-42af-8348-4884d8539b81 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.693236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa19fc-0da1-4ca9-b0c8-db353987c376 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.694154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 222/456] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:38.694263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 235/457] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:38.699381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415c13-2b5b-436d-aadf-9430785f81cf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.699533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0d4a-3ad8-4160-af2a-9633732404c3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.699681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415c13-2b5b-436d-aadf-9430785f81cf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.699788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0d4a-3ad8-4160-af2a-9633732404c3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.699934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415c13-2b5b-436d-aadf-9430785f81cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.700026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0d4a-3ad8-4160-af2a-9633732404c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.720953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-64415c13-2b5b-436d-aadf-9430785f81cf None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:38.722140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64415c13-2b5b-436d-aadf-9430785f81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:38.727511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f21d0d4a-3ad8-4160-af2a-9633732404c3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:38.728669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f21d0d4a-3ad8-4160-af2a-9633732404c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:38.730291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:38.811598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 236/458] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:38.858062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 223/459] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:39.173332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003961c8-bab0-4463-80d0-48bb9eecad7a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.173684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003961c8-bab0-4463-80d0-48bb9eecad7a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.174058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003961c8-bab0-4463-80d0-48bb9eecad7a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.176208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 237/460] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:39.181335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.181627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.181904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.188599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003961c8-bab0-4463-80d0-48bb9eecad7a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.212362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:39.213739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:39.314560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 224/461] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:39.315529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.333761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003961c8-bab0-4463-80d0-48bb9eecad7a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.387688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.404544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.406720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.407387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.407800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.423104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.424869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.432622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7b683371-8179-4881-920f-45baa9424f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:39.447734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:39.448701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:39.449541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:39.449541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.449541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.452453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 238/462] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:39.454103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f4f-20a1-42a2-a035-b013127e2b2c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.454569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f4f-20a1-42a2-a035-b013127e2b2c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.455008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f4f-20a1-42a2-a035-b013127e2b2c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.456844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 239/463] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:39.463554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e46f4f-20a1-42a2-a035-b013127e2b2c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.477643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127da80-174b-4924-ae0e-a1d3cc592dd2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.520586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.521318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.521688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.522031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.533659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.535757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bbe30273-df21-4689-9cc7-e83705a8ea32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:39.553990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:39.554979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:39.555819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:39.555819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.555819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:39.558509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 225/464] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:39.559889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.560340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.561016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.582867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:39.584956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:39.587050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e46f4f-20a1-42a2-a035-b013127e2b2c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.630382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.631345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.631627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.631897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.633136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.633460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.635020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.654246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 226/465] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:39.655153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.664971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 240/466] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:39.666404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.725439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.751271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.786764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1f60-f5aa-4acb-88a9-1416805462da None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.824087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.824609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f8b8c-b946-4e4b-a54f-d7f60f281c2a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.825292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.825782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.826245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.827097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.827408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.828786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.845464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 227/467] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:39.846178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.860429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.861053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.861318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.861585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.873887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.875377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-58557f44-a70d-4f95-8a23-8e18ee43e34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:39.890732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:39.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 241/468] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbde54-f423-4985-b1aa-8c4ee0a0d050 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbde54-f423-4985-b1aa-8c4ee0a0d050 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbde54-f423-4985-b1aa-8c4ee0a0d050 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 242/469] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:39.902266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbde54-f423-4985-b1aa-8c4ee0a0d050 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.958299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6888a1-39dd-4097-9705-6f9f93ca9fe2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.971458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.993498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.994162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.994832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.995173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.995468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.996187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.997610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.003410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbde54-f423-4985-b1aa-8c4ee0a0d050 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.018213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 228/470] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:40.019080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.041179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.041853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.042135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.042479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.043145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.044767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e4b07-14e9-4e93-b6bf-cc0cf189bd2a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.068111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 243/471] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/users => generated 248 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:40.069378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.069662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.069934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.093199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:40.094565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:40.179682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c871d-f6de-4f04-b51e-6bee431a1b7c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.202943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 244/472] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:40.204214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.239291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.240239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.240717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.240998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.241666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:40.241975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.243422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.267306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 229/473] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:40.273496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.310253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:40.321543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:40.360820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92b25-d23c-4363-949e-140da1b43099 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.403212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dac9f-d265-4007-95de-3c39109358a7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.412871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.413769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.414204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.414699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.429812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.431883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.447479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.448309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.448808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.449149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.450042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.451685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:40.454780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:40.455677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:40.456560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:40.456560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:40.456560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:40.458444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 245/474] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:40.464770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.484746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 230/475] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:40 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:40.527695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.596963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d982-ce70-4041-bf7c-76b8595abce3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.631503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:40.637983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.638633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.638918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.639193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.639854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:40.640204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.641578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.647902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:40.656139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 246/476] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:40.662102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.668485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4742a-5b93-48c8-94c0-a147b6260fdd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.708552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.709176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.709453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.709723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.710402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.711792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.719902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:29:40.719902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 15:29:40.719902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [parameters: {'id': 'ea49d07dcdd84da783ec93f17f295ac9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 15:29:40.719902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:40.739928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return ref.to_dict() Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:40.741352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:29:40.742294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:29:40.743329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:40.744229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:40.745057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'id': 'ea49d07dcdd84da783ec93f17f295ac9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:40.745854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 15:29:40.746893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.747745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 231/477] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 15:29:40 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:40.750465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:40.824796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916d7e-9826-416b-853a-b487afb1defa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.865175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.865899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.866186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.866488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.867204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.868657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.879313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead90d88-115e-40af-b6d4-82b086836d8c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.895118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 247/478] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:40 2022] POST /identity/v3/users => generated 249 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:40.924250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.925051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.926106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.926539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.939734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-46868beb-2ad6-40e6-8faa-3d458c60e781 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:40.945770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:40.946865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:40.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 232/479] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:40.955067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:41.022033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:41.109403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddc1f-5626-48f4-b409-4a1899398126 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:41.149291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:41.150320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:41.150825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:41.151315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:41.152191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:41.152693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:41.154276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304bbe0-3fe7-470e-b69d-d3189a6f8340 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:41.164368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 248/480] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:42.735678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552262e-3c08-41b6-86f8-0ebf27b81dca None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.736185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552262e-3c08-41b6-86f8-0ebf27b81dca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.736261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552262e-3c08-41b6-86f8-0ebf27b81dca None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.738379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 233/481] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:42.743877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc7eb-4db8-4f52-a525-b37639818fc6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.744328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc7eb-4db8-4f52-a525-b37639818fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.744402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc7eb-4db8-4f52-a525-b37639818fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.766456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4edcc7eb-4db8-4f52-a525-b37639818fc6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:42.767745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4edcc7eb-4db8-4f52-a525-b37639818fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:42.859486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 249/482] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:43.046894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01044c84-771c-4fee-bca2-b25a8a9cd26a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.047531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01044c84-771c-4fee-bca2-b25a8a9cd26a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.048020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01044c84-771c-4fee-bca2-b25a8a9cd26a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.049790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 234/483] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:43.064022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.064303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.064575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.091388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:43.092574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:43.129711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.183955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 250/484] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:43.195084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01044c84-771c-4fee-bca2-b25a8a9cd26a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.208608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.308078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01044c84-771c-4fee-bca2-b25a8a9cd26a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.349101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.349887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.350174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.350462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.361927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-52dfa0fe-8eb3-46d8-ab53-950f514d179d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:43.367522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:43.368410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:43.369624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 235/485] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:43.374754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.437367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcceb7-5c0d-4e22-8c7f-1da39d69ad57 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.437998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcceb7-5c0d-4e22-8c7f-1da39d69ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.438515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcceb7-5c0d-4e22-8c7f-1da39d69ad57 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.440041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 236/486] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:43.445435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.445954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.446477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.470649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:43.473682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:43.556008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.566616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 237/487] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:43.567819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.574435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ea6-abdb-4988-8515-5761b3819581 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.610869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.611864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.612139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.612454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.613180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:43.613483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.614895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.624535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 251/488] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:43.630223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.666245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.787102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213053d-ed4f-452f-9523-f50213d06848 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.788705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f8ce-20dd-4269-83fb-5182e833d326 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.836926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.837475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.837748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.838016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.852880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.853501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.853815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.854215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.858862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:43.869737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:43.870730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:43.871941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 238/489] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:43.875354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.876939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.879553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:43.906452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:43.907467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:43.908310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:43.908310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:43.908310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:43.912106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 252/490] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:43.913581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.959649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.004933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702657-e0ff-4c98-9255-eca5e09a2c54 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.012563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.031271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539a51b-85cd-499b-b9cc-468417ec0a16 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.042508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.043111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.043389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.043659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.044318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.044608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.045993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.053857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 239/491] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:44.054705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.064617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.065241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.065515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.066310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.076667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-de0b4831-9a6f-452b-b22e-03557a71b865 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:44.083202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.084200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:44.085016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 253/492] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.085970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5babbf4-b9e4-4917-bc07-07551b7412c1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.085970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5babbf4-b9e4-4917-bc07-07551b7412c1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.086324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5babbf4-b9e4-4917-bc07-07551b7412c1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.087512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 254/493] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:44.088436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8063-3451-4468-a7b0-c431a96caf0f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.088749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8063-3451-4468-a7b0-c431a96caf0f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.089151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8063-3451-4468-a7b0-c431a96caf0f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.094195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 255/494] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:44.094978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b8063-3451-4468-a7b0-c431a96caf0f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.163086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705d83a-8c27-4eba-a975-bfcb906211c2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.184347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.195703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b8063-3451-4468-a7b0-c431a96caf0f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.197075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.197678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.197961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.198234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.198918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.199208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.200572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.212674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 240/495] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:44.213485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.234121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.234771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.235044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.235312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.245388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.246876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.248789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ad6e94bc-bc56-404a-bd49-0dca22a5ced1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:44.264103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.265003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:44.266273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:44.266273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.266273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:44.266273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 256/496] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.271010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ca1e5-6e3b-41d2-8edb-4766b66b483a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.271245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ca1e5-6e3b-41d2-8edb-4766b66b483a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.271520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ca1e5-6e3b-41d2-8edb-4766b66b483a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.288913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-250ca1e5-6e3b-41d2-8edb-4766b66b483a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:44.290098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-250ca1e5-6e3b-41d2-8edb-4766b66b483a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:44.312966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed438-a068-457c-bcbf-9a9ec07fac12 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.346323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.346884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.347215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.347651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.348337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.348643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.350251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.355276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 257/497] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:44.356401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.356697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.356973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.363248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 241/498] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:44.363248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.381190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:44.382367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:44.410933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.452661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 258/499] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:44.453596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.475461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef26fb-a821-4990-9882-65743c42a5e8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.512833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.513463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.513741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.514014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.515240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.522773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.524116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:44.549758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.552094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 242/500] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.553712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.584815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb567b75-9276-4e57-9fa7-7e1f4b8f9f01 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.621539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.622059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.622335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.622630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.623368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.623674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.625064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.637204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 259/501] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:44.638001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.677396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597890d-aa10-4db3-b045-0d150de5c0f9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.681237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.717990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.718657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.718940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.719213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.731100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.732573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:44.747561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.748496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:44.749359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:44.749359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.749359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:44.749359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 243/502] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.750594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.751547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aaffd-920c-456c-a806-a47c4d666e14 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.766303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.786792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.787392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.787668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.787940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.788658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.788969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.790390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.827338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 260/503] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:44.828185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.856084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb25e-d755-485a-8ce5-faa796adf2d3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.890543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.891024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.891303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.891575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.902303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.903858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:44.918229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.919326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:44.920163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:44.920163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.920163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:44.920163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 244/504] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.920163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.934583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52eb9-da53-4ca4-b047-a683f80f838f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.938550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.969044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.969669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.969942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.970208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.977559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.979047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:44.992247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.993235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 261/505] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.994205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.024729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.028933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb72e-2c51-4d5e-bf9d-bf2865068e0e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.061474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.062087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.062362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.062653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.072718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:45.077902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:45.078988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:45.079789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 245/506] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:45.080442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.093711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5e07-4250-422a-8457-8b1395f18042 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.127893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.128497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.128780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.129056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.129695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.130003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.131429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.142454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 262/507] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.143188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.146770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.180968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ec3-f831-40c8-a107-d97bc0939097 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.213620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.214543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.214821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.215089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.234088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.243384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d0ec0-ba6e-4337-8ca3-c6809c54a1d4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.279495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.280098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.280376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.280648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.281286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.281588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.283006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.283259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.285426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.296465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 263/508] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.297276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.351011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.387101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 246/509] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:45.387728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.399338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42d1ab-aba4-40e4-b7d6-cb92f977e1b4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.433984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.434691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.434972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.435253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.436221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.436526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.437947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.455566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 264/510] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.456933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.469959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.484655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccba64-2144-4379-884f-5a6b8cc2505a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.517547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.518147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.518450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.518730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.519924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.520225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.521606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.529463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 247/511] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.530376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.560321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2a67-7098-44ef-9b06-95e5cae7e1e2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.560790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.595062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.595685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.595967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.596243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.603282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.604732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:45.616170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:45.617102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 265/512] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:45.617978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.634146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224038de-a8ce-4670-9f16-5c7c16c70cd6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.667362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.667904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.668185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.668458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.669102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.670554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.686902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.692552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 248/513] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:45 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:45.693371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.727047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cadd6-4bed-411e-98ab-dd55754e55cf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.765481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.766267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.766573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.766703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.777685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.779187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:45.794924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:45.796368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:45.797197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:45.797197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:45.797197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.797197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 266/514] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:45.797197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.806798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.827535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0ef18-ef92-45b9-8f5d-6e2c32e23386 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.864901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.865657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.865824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.866053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.913741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408301-d3ea-4faa-b3a9-892915741d63 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.939274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.951975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.952659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.952941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.953207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.953899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.955364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.963696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.965093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:45.980344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:45.981283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:45.982110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:45.982110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:45.982110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.982110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 267/515] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:45.982334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.047522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.067063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 249/516] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:46.072145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.084815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b718-8505-4975-a776-678c7a0edc42 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.118218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.118848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.119132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.119412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.120147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:46.120457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.121911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.143657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 268/517] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:46.143657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.172930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.188023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e091-3005-4a15-829c-75a3434f6397 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.229603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.229900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.230175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.230472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.231123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:46.231428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.233095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.247400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 250/518] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:46.247587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7486f03-6377-4164-8501-781b0a3c7c77 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.247676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.284234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.285355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.285638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.285906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.286591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:46.286896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.288262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.299590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 269/519] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:46.301093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.308036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.370261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3340-d45c-40eb-8d09-e1aec291d079 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.392616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.402230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578f914-fb7f-41a5-a4d9-4dba7e3459f0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.410689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.411465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.411741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.412009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.423460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.424907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.436168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.436772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.437047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.437310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:46.440355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:46.441712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:46.447673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:46.447673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:46.447673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:46.447673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 251/520] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:46.447673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.517962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.519426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.548817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.554711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba160-65be-42f8-bcbe-56461992a3f3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.592753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.593699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.594156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.594646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.601816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.603528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.615615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:46.616892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:46.617726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:46.618745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:46.620882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 252/521] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:46.621828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.632374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 270/522] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:46.633781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.743209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c809a3-6e23-44bb-befd-0342b136bd5c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.743703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3a428-4843-4582-9393-9f06ee4dc23c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.777062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.777574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.777957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.778151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.778773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:46.779073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.780440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.780960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.781614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.781969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.782242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.790147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.796087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 271/523] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:46.796548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.796640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:46.808280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:46.809203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:46.810102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 253/524] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:46.810958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.832379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.852247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:46.903316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f471f7-9787-4e97-af12-f34d61849a49 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.926555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323bfd7-938e-40a3-ba90-a0bfa391ac31 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:46.939854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.940463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.940777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.941048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.941740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:46.942047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.943521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.961831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 272/525] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:46.962610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:46.967822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:46.968473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:46.968760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:46.969065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:46.981339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:46.982841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:46.999878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:47.000891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:47.001744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:47.001744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:47.001744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:47.001744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 254/526] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:47.001744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.069536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960808-0ee5-4b0a-ae45-d70d5b2c15be None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.075327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.082630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.104429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.105058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.105338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.105610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.109425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0a16-bc22-4f20-976b-902b1bc05630 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.112153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.114175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:47.126064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:47.127085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:47.127895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 273/527] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:47.128415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.153840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.154489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.154773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.155048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.155778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.156083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.157517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.197444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 255/528] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.198599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.235692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543ade2-5312-4f7e-9cd0-93b5362492c2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.276007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.276717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.277000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.277273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.278053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.278294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.279886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.294444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 274/529] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.294444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.322702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21ad9c-3a56-4005-9768-79a9f55164ee None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.333525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.362174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.373008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.373817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.374190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.374639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.375370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.375791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.377296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.395920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 256/530] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.397032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.433810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5115c76-a2dc-468f-87fe-33f7f46e621a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.472457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.473336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.473722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.474106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.474958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.475366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.476926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.495845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 275/531] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.496985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.535492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae288-1a3c-4761-b699-f597010d3ede None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.576362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.576887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.577164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.577435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.607333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.614258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fa5de-1008-4033-8099-ad06413beb6f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.647300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.656885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.657554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.658069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.658169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.667511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.668997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.669685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.672644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:47.689318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:47.690189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:47.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:47.693955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 276/532] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:47.695250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.784868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 257/533] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:47.786673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.824428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162b0b8-f651-40ce-ada3-5b25c2e11dfe None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.864887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.865697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.866000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.866492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.867371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=25cbe9fdae7b41ad9735c20260ffe22f) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.867713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.869304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.881245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 277/534] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/role_assignments?user.id=25cbe9fdae7b41ad9735c20260ffe22f&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.882387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.910203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.927934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f841e-db6d-4faf-905d-de511557ed37 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.932701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.981109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.982715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.983294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.983822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.984936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.985855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.988409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1415511-c243-4af0-b8e0-b507f9b6b936 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.015187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 258/535] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.017677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f44f-dfcd-4d35-92a0-8a91d0e47eec None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.018121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f44f-dfcd-4d35-92a0-8a91d0e47eec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.018560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f44f-dfcd-4d35-92a0-8a91d0e47eec None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.021804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 259/536] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:48.023069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02f44f-dfcd-4d35-92a0-8a91d0e47eec None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.034026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69915aaf-a63d-4484-aa2c-2771d936240c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.083919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.085331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.085899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.086464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.101259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.103328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-dbf0f75a-1485-481c-bb50-72864a78c880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:48.125602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.126658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:48.127500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:48.127500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:48.127500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:48.127774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 278/537] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:48.129033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.129313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.129581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.151535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02f44f-dfcd-4d35-92a0-8a91d0e47eec None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.152239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:48.153615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:48.190808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.191839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.192385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.192664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.193503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=9685e7e865604d4b9acd36902045d141) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.193819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.195383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.202872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 260/538] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/role_assignments?user.id=9685e7e865604d4b9acd36902045d141&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.203772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.224685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.236215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 279/539] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:48.332406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1ad08-139d-4cb5-9cc9-f53306cd0b2f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.373778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.374405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.374708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.374981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.375691 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.375938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.377330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.397279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 261/540] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.402618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.405844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.528368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0248a81-1585-4fd9-a0f8-f86861665e32 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.575693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.576332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.576618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.576895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.587704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.589277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:48.604348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.605215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:48.606042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 280/541] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:48.612285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.631323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.696539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.731546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe9768-8995-408c-8868-6c33724179b2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.771123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.771736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.772016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.772290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.772989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.773295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.775982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.791998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.795840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 262/542] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.803398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.866323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87a6d9-684d-43ee-9707-c9f8646b0d81 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.951637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fada-6f9a-490a-83c0-6d7f2cdc2d12 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.964603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.965254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.965537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.965813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.986542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.987251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.988092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.988373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.989105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=1393d3a8c9174d08b2c6d63cf4e25b19) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.989416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.994667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.995098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.000354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:49.001244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 263/543] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/role_assignments?user.id=1393d3a8c9174d08b2c6d63cf4e25b19&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:49.009975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:49.011127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:49.012115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 281/544] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:49.019288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.181770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639672-6cec-4dc7-bc7d-019348caa22b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.233988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.234848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.235273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.235719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.236524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:49.236970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.239152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.253913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.254530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 264/545] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.442402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea345c-e800-4c14-a921-c8d82f8f7b7b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.517763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.562298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.562974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.563249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.563527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.584287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.585755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:49.612014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:49.612998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:49.613837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:49.613837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:49.613837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:49.613837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 282/546] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:49.620618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.730783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.796400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0813-7bc5-40d6-9f68-3c87d92507f8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.882537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.883199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.883531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.883814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.884469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:49.884829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.886210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1beb-30b9-4a88-9a33-c3dca20755b4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.908362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 265/547] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.915874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.994683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a1899-12bf-4efb-a665-63f1751baaca None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.995192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a1899-12bf-4efb-a665-63f1751baaca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.995271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a1899-12bf-4efb-a665-63f1751baaca None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.996634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 266/548] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:50.003732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.004019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.004279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.043429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:50.044639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:50.153002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f621e2-64b7-4911-a8ef-162e649e1b1b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:50.195966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 267/549] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:50.210003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:50.211067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.211553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.212028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.221242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:50.223627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:50.236051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:50.238082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:50.239239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:50.241815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 283/550] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:50.248138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:50.314058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:50.382873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52714e-eb18-4daf-b2bb-6902dd8bc241 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:50.431266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:50.432226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.432704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.433214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.434156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:50.434697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:50.436578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00181dc-4cdc-4652-a10f-6c71ec70a532 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:50.459640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 268/551] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:29:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:50.467514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:50.479396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7094dd-3b90-4e4f-8d28-1a76dd58cf04 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.480209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7094dd-3b90-4e4f-8d28-1a76dd58cf04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.482315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7094dd-3b90-4e4f-8d28-1a76dd58cf04 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.484082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 269/552] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:50.489448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.489968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.490527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.523685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:50.526009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:50.529612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:50.619130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 270/553] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:50.620384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:50.629193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787505af-b328-4adb-8a3e-aa06d9e2ef97 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:50.656459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:50.719233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:50.719775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.720057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.720348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.740309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77e02-b783-46ef-8c2b-cca72d5bbc64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:50.780301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:50.781320 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:50.781883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:50.782443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:50.793909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:50.800153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:50.801106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:50.801920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:50.804369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 271/554] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:50.812380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:50.873470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:50.877132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:50.952774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.000050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.010353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6993a20-a653-44e2-ae6f-9dd78b65306e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.025746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 284/555] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:51.026787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.050769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.051890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.052706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.053532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.054445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:51.054968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.056541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.067279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 272/556] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:51.068466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.145759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9c89-18f5-4d12-a326-16c992da7943 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.180029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.180655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.180929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.181195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.183352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.184349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95f867-363b-469d-801b-b5d1cc44cb3e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.194506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:51.199790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.200725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 285/557] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.201643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.225913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.226560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.226853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.227423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.239080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.240256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-66716bc7-32db-495b-a7dd-6c9f2d7742c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:51.261919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.262937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 273/558] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2de1-7cc5-4ac7-bb0b-f9ac1f680c4c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2de1-7cc5-4ac7-bb0b-f9ac1f680c4c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2de1-7cc5-4ac7-bb0b-f9ac1f680c4c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 274/559] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:51.267244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f2de1-7cc5-4ac7-bb0b-f9ac1f680c4c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.279172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.323576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66290b2-e35e-4a3b-a915-c9bfd69ee9d4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.358247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.358920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.359205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.359481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.370964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.372553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.375772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f2de1-7cc5-4ac7-bb0b-f9ac1f680c4c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:51.388018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.389297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:51.390366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:51.390366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.390366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:51.394769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 286/560] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.395983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.415794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.416486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.416676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.416943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.417606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:51.418081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.419323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d576c-e357-47ef-a534-0d6df15e79e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.428293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 275/561] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:51.429366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.429653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.429926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.447651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:51.448827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:51.455556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.513578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e20b-1773-4527-99b7-2a581ac71547 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.522586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 276/562] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:51.523933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.526553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.562531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.563487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.563961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.564416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.565392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:51.565977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.567651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.580658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 287/563] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:51.581938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.641111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc159e54-f0f4-42d9-b90b-81036841467f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.676586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.677197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.677476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.677750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.678391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:51.678884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.681040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.693863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 277/564] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:51.694684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.716220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd897d-ce7f-4708-831b-ada18f349526 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.725616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.754431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539a232-51e3-441b-9553-391098695909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.755077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539a232-51e3-441b-9553-391098695909 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.755360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539a232-51e3-441b-9553-391098695909 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.755633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539a232-51e3-441b-9553-391098695909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.760949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.765942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539a232-51e3-441b-9553-391098695909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.767368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539a232-51e3-441b-9553-391098695909 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a539a232-51e3-441b-9553-391098695909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:51.783061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.784142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:51.785536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:51.785536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:51.785536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:51.785536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 288/565] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.786715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f3645-3183-49c7-af1c-477d851dde5e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.786715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f3645-3183-49c7-af1c-477d851dde5e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.786715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f3645-3183-49c7-af1c-477d851dde5e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.787887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 289/566] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:51.791186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f3645-3183-49c7-af1c-477d851dde5e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.807680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac819bb-76f7-42fa-ba4c-6d4e500ca050 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.843543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.844172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.844450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.844723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.852329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.853777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:51.865029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.866124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:51.867254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:51.870793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 278/567] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.871617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.902707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f3645-3183-49c7-af1c-477d851dde5e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.939497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.940125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.940410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.940684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.947707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:51.949150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f499c738-c8cf-4f2f-8a84-1e1a2159e67a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:51.961289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.962851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 290/568] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.963781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.964521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.973709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016aa35-ff49-45f8-af0a-07974f0073b7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.989235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:51.990538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:51.998727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.009941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.013009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.013616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.013891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.014157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.014820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.015123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.016527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.028351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 279/569] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.029142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.068799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 291/570] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:52.069658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.142289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aebb9-65ee-4eb4-b0c4-1d080faee5bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.172524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12263de5-5d40-48ef-ad2e-69973cde40a9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.178165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.178821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.179102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.179375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.189837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:52.195200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.196117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:52.196975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 280/571] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.197705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.212039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.212962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.213523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.213800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.214522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.214974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.216929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.235908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 292/572] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.235908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.246548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.247392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.309903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819a7b4-2938-443a-a3da-03bb4c0467bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.344514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.345302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.345419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.345674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.346484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.346790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.348234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.352440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac7a88-c490-4da4-b138-0d9e8cc92f37 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.368246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 281/573] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.368919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.390663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.391295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.391576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.391846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.399702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.401134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:52.412123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.413320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 293/574] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.414671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.473048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.476655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041738-f6e6-4521-8661-057fbd76a65f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.482841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.511772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.512473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.512826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.513104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.513796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.514101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.515513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.518893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fe006-d3e8-4c0c-b166-3599c34742cf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.523997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 282/575] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.524791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.552618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.553260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.553546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.553827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.554617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=8daf1537e6fa46238223fd03ced27a43) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.554932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.556375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.562121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 294/576] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/role_assignments?user.id=8daf1537e6fa46238223fd03ced27a43&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.562859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.642696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d486d46-16b4-4b21-9d7f-82f6d49aca44 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.678706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.678970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e75a35-60ce-4b84-9602-559662ff58dd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.679348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.679624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.679896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.695164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:52.700750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.701729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:52.702618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 283/577] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.704636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.717053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.723863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.725057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.725463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.725820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.731132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.740833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.742407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:52.757557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.758760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:52.759837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:52.759837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.759837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:52.760574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 295/578] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.761352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.820173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a68a4-3e72-41d1-9e3b-371877627a71 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.855449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.856200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.856609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.856985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.857815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.858243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.859824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.868028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b4e-8e64-4404-bf4e-5e56ce72e8fd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.885750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 284/579] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.885750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.909469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.910099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.910401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.910698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.921429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.922921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:52.939467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.940716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:52.941873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:52.941873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.941873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:52.942891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 296/580] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.943782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.983197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.984261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.006244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d965d-5cb2-4dc5-8d44-013861fb1877 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.044652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.045422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.045799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.046174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.047040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.047345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.048707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.052699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68aefd-900f-4112-a803-6c42b366fef2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.057719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 285/581] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.059387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.090143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.090733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.091009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.091284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.091921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.092232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.093629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.110752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 297/582] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.110752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.172291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599c50-f491-400c-a34b-531e860b552e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.208838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.209806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.210608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.210770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.211062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.216656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c353d-d99d-4b87-8997-68b380f3c67c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.223313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.224765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.226737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:53.241443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.242524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:53.243426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:53.243426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.243426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:53.243426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 286/583] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.243426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.263041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.263796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.263932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.264208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.264863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.265234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.266627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.283825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 298/584] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.283825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.365458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3c351-bcbc-4910-80a7-e58b49c9de3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.400787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.401421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.401702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.401975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.408160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72f8b-c4ef-4d04-8cea-4dc6a9ac44a7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.413521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.415053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:53.431037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.432051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:53.432945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:53.432945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.432945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:53.432945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 287/585] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.432945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.453600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.454628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.455118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.455597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.464660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.466152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:53.477552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.478774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:53.479660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:53.483513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 299/586] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.484144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.488463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.506010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.542602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09e83f-0350-41b9-8d63-92b7ae4eac0f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.578952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.579560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.579837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.580108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.580770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.581091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.582624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.593030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e9817-fb9d-47c0-b825-eec0a8971734 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.594265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 288/587] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.595144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.638671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.638671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.638671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.638671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.642026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.643555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:53.658232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.659899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:53.660980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 300/588] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.662055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.720688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569a1a2-ccd1-45f9-99ef-1a6218ddc2af None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.729503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.748317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.762430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.763124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.763452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.763763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.764409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.764797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.766085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.773040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46928e85-6f62-42c5-97ed-b5169616ae67 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.778054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 289/589] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.779145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.810602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.811632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.812124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.812472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.813223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.813532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.815740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.827770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 301/590] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.833715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.893247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cb58d-39d6-4b94-87ba-46ee49ee027b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.930549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.931173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.931447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.931719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.939471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.940941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.949358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09344d-09cd-4fde-ad0a-8c08fa7f6209 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:53.957685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.959086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:53.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:53.963547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 290/591] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.964215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.989205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.989883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.990176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.990351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.991109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.991418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.992848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.994138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.996161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.011462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 302/592] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.012252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.083763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859f0c-39ba-4653-9e75-e383f0a85f77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.123298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.123926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.124205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.124477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.127119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428f95-e20f-4b45-9dc5-cdf78915f4e9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.132291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.133726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:54.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:54.146739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 291/593] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:54.147780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.171411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.172325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.172758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.173193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.188970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=6fcc9eec4e114f538c506178c744b49a, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.190690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.205074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 303/594] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:54 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:54.215531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.270275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.291731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.292381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28ca1-9c1f-4ca6-a745-c3d25c21fe48 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.336750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.337331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.337611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.337880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.338616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:54.338920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.340431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.356780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 292/595] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.356780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.369463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958abe-a41f-4e30-894d-8cc037e8aaf8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.417156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.417517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.417760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.418033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.433178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.433178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:54.448641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:54.449636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:54.450584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:54.450584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:54.450584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:54.450584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 304/596] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:54.450822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.490758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4d48-91ef-4bdf-b761-aba04a531d4b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.531940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.532556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.532839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.533111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.534441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:54.534764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.536671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.557159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.565869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 293/597] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.565869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.570160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.572243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6c9ba-d026-4ace-be53-393f8646c6de None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.612932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.614033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.614564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.615124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/25cbe9fdae7b41ad9735c20260ffe22f/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.632212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=25cbe9fdae7b41ad9735c20260ffe22f, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.634326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.653279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 305/598] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:54 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/25cbe9fdae7b41ad9735c20260ffe22f/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:54.655231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.685365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5875c2-bb1c-41d9-84c5-815e18fec95b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.725045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.725402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.725687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.726045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.726741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:54.727047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.728472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.741951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 294/599] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.747520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.783458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d3c1-2e5d-4253-b25e-cce6051d404f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.822946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.827292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.828343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.828845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.829321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.841916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.843860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.852989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d30aabe4-ebf3-4097-8fc3-024845050aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:54.863713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:54.864747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:54.865629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:54.865629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:54.865629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:54.872585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 306/600] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:54.873504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4550-5865-40e8-84f7-77605b17e5ec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.874693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee97bb-47d6-4482-a64e-97bce4ea134d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.875246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee97bb-47d6-4482-a64e-97bce4ea134d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.875777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee97bb-47d6-4482-a64e-97bce4ea134d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.878712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 307/601] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:54.879146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee97bb-47d6-4482-a64e-97bce4ea134d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.937340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.938810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.939315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.940883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.950874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.953208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-68ec3807-d161-4e1b-98e9-afeb4b59b7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:54.969387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:54.970334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:54.971249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:54.977847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 295/602] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:54.977847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.977847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.977847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.998911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:55.000725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:55.012660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee97bb-47d6-4482-a64e-97bce4ea134d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.048406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.049153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.050383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.050771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.051622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:55.052015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.053525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.067186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 308/603] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:55.068150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.086017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 296/604] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:55.087583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.149584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.187926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.203839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6bd79-7214-46d6-b2b0-7621b56c9d96 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.259300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.260267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.260716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.262180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.276966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.278494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.286656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d3866-71aa-437d-8597-3d85ea25c3af None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:55.311709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:55.313243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:55.315508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 297/605] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:55.324349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.380155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.381036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.381452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.381886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.383037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:55.383479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.385101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.408702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 309/606] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:55.416226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.477264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0275e-46e9-497b-81b1-d245692c6d40 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.494208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.520622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.521968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.522549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.522974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.523973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:55.524434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.526773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.545195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 298/607] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:55.546277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.554312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8944-1e83-401d-84f6-cbed56ba4e74 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.577256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.601088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.601901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.602284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.603184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/9685e7e865604d4b9acd36902045d141/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.623805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=9685e7e865604d4b9acd36902045d141, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.625391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.653140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 310/608] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:55 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/9685e7e865604d4b9acd36902045d141/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:55.655429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.688022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ced00b-dc39-4bf5-b1ad-2699f97eb3ca None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.795303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.795896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.796194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.796454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.820156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ae4f-fb75-4521-bd62-6ee2231ed293 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.831809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:55.836850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:55.837797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:55.839244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 299/609] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:55.844046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.876331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.878922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.879212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.879487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/1393d3a8c9174d08b2c6d63cf4e25b19/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.917918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=1393d3a8c9174d08b2c6d63cf4e25b19, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.920118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.922096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.948610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 311/610] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:55 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/1393d3a8c9174d08b2c6d63cf4e25b19/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:56.011754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8524d6-e6f3-491a-b08b-baae3327ca15 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.056500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.057037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.057385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.057672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.058342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:56.058717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.060458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.073350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 300/611] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:56.078740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.101156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:56.208529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cfa1-45af-48bc-96f7-e77ff2dd349a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.265554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.266237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.266631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.266909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.279229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.280812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:56.298302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:56.299409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:56.300826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:56.300826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:56.300826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:56.300826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 312/612] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:56.305903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.422837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:56.430207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6916a02-3465-42bb-b839-321c5994b898 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.478006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.478650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.478932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.479207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.479859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:56.480163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.481532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.494579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 301/613] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:56.499782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.591805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:56.641537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c333-d84f-40c3-8ee9-4379b81deca4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.690667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.691235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.691517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.691787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.703558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.705077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0263b60d-170a-4f83-88f6-06b3de7ccc05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:56.719491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:56.720522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:56.721521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:56.723839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 313/614] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:56.731974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.751773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119313-6077-4f58-a2dc-88daf4c1c260 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.752930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119313-6077-4f58-a2dc-88daf4c1c260 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.753415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119313-6077-4f58-a2dc-88daf4c1c260 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.755308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 314/615] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:56.760691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.761216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.761719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.781087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:56.782689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:56.858534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 315/616] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:56.934192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735453bc-1ca5-4558-8ec3-8556883c1d43 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.984522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.985058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.985337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.985612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.986342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:56.986770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.988713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d864c55-b190-45dc-a72b-36fbb6fa6333 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:57.015437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 302/617] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:57.023518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.145178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.170357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7168c-cdc7-4356-9fdd-f124b06539ff None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.170718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7168c-cdc7-4356-9fdd-f124b06539ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.170951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7168c-cdc7-4356-9fdd-f124b06539ff None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.172556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 303/618] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:57.177904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.178193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.178498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.205171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:57.206946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:57.226251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.228476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8286ec5-a7bd-48b8-b360-ad411bf38d54 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.289051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.289716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.289991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.290260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.305997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:57.307776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:57.314028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 304/619] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:57.323235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:57.326199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:57.327248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:57.328129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:57.328129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:57.328129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:57.328129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 316/620] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:57.334117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.462377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f4ac-4076-4b5d-928a-a2f9afbde769 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.477921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.502869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.503940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.504969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.505393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.506337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:57.506985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:57.508755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:57.529080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 317/621] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:57.536070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.578323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fb6fb-ea3e-4dea-8b1c-7a2cdc5b07f4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.642448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.643118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.643392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.643658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.654952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:57.661059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:57.662212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:57.663160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 305/622] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:57.668019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.676583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.720320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff660-dc2f-4927-9ba1-31b22e1ed26a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.759521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.760264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.760643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.760923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.770471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:57.772106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:57.781467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69ef91-a565-40d1-8c31-06a721c24f3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:57.785804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:57.787840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 318/623] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:57.790058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.823052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.823317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.823780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.824194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.824487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.825367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:57.825702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:57.829718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:57.842502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 306/624] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:57.843705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:57.930701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:57.932633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f9cca-29aa-4d68-803c-d6c10252a14c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.972814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:57.973828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:57.974427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:57.974904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:57.981784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850d92-456a-4966-8677-b30e969934d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:57.987251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:57.993238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:57.994211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:57.995081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:58.002597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 319/625] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:58.003944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.022770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.023512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.023792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.024063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.024751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:58.025052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.027155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.045288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 307/626] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.046180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.114560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.131076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2231d7-cf55-46b9-a9de-31bb47a4275b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.180607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.181308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.181614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.181807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.188002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c1e2-69ce-4faa-b707-e526c72a3ee0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.196304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.197806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:58.214900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:58.215932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:58.216843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:58.216843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:58.216843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:58.221260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 320/627] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:58.221851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.224548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.235910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.236983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.237499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.237975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.238871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:58.239385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.241114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.256234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 308/628] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.257525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.334256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8bd2d-731c-4f7e-a35c-999e9c7eb576 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.365700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380d2-da41-421f-acff-ce6aef650c18 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.371132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.372064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.372675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.373021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/8daf1537e6fa46238223fd03ced27a43/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.389670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=8daf1537e6fa46238223fd03ced27a43, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.391174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.394302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.402839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.403670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.403949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.404222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.404856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:58.405145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.406533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acdac6-523b-4c12-bb03-8ed1b7f6383a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.408061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 321/629] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:58 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/8daf1537e6fa46238223fd03ced27a43/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:58.409050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.439408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 309/630] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.441832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563862-f29a-4849-918f-67c480e662bc None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.442441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563862-f29a-4849-918f-67c480e662bc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.442892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563862-f29a-4849-918f-67c480e662bc None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.444900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 310/631] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:58.446136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f563862-f29a-4849-918f-67c480e662bc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.538189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.604981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721afe7-1063-4a7f-be4f-3451559bb380 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.614609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f563862-f29a-4849-918f-67c480e662bc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.655574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.656490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.656771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.657044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.660027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.660660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.660940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.661209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.668783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.669072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.670353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.670508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e022717b-e67b-43ea-bf88-43c53399f56d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:58.683450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:58.684465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:58.686113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 311/632] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:58.686748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.686748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.687148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:58.688508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:58.691216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:58.693337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:58.693337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:58.693337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:58.693337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 322/633] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:58.698231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.716409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:58.716736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:58.758602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.787791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 312/634] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:58.788706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.825908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593450-6491-4682-8bac-42f9e26bd4f4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.836543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:58.875102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.875629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.875907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.876176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.876826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:58.877128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.879380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.894075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 323/635] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.900707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.961742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8940-313e-4ffe-b9ff-b4b9624a4f31 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.997756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.998687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.998968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.999238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.000027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:59.000369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.001820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f9896-6904-474f-86b4-be411a3a8de8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.013218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b629ff-b2db-400d-a567-e7f406688bac None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.024913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 313/636] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:59.026370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748441e2-8f6d-4f64-a97d-16ba70bcef3d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.026679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748441e2-8f6d-4f64-a97d-16ba70bcef3d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.026956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748441e2-8f6d-4f64-a97d-16ba70bcef3d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.028792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 314/637] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:59.029803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748441e2-8f6d-4f64-a97d-16ba70bcef3d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.040475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.052602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.053338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.053629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.053898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.062635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.064206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9affd92e-4b1c-4b9c-b39b-dfcf9a0bf085 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:59.077816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:59.080059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 324/638] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.082325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.084074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.113572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:59.115193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:59.145807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.156368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748441e2-8f6d-4f64-a97d-16ba70bcef3d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.198754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 325/639] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:59.200217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.209815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.210671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.211044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.211412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.295408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.297234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.333900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.340900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c4980-723a-4f0c-86cd-d2abe01d74d0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.378424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.379214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.379591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.379961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.380789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:59.381195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.382917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.407551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 326/640] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:59.408766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.420545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.438455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 315/641] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:29:59.440422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.529932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b042e-c838-4cca-a829-4c6edcee0850 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.563396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d35f1-750b-4da4-9455-77c3ddeba5fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.566848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.567501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.567680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.567897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.580706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:59.586538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:59.587477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 327/642] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:59.590990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.601428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.602034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.602304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.602603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.610238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.673099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.680637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.682221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.703606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53ce2-cc41-4b33-97fb-a05d88fe63bb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.739995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.740998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.741277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.741546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.752142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.754266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:59.769476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:59.771164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:59.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:59.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:59.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:59.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 328/643] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:59.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.806125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 316/644] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:29:59.807294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.851349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.891960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cc1a-1318-49ff-be85-639d5e7e9bb2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.923745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.931931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.932671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9952d-3a0d-4a25-8111-73ddc0036d82 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.933553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.934130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.934713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.946278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:59.952513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:59.953438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:59.954433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:59.955503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 329/645] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:59.956838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.976301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.977183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.977671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.978083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.979050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:59.979499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.980876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.995142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 317/646] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:59.995142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ca47e-756b-45db-8738-5d26e0569275 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.097100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.106313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.106957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.107221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.107492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.108124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.108425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.109809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.111919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82bff-a92b-48cb-b5d0-36d5a13bf949 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.123713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 330/647] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.124548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.154000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.155235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.155690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.156135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.168455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.170282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:00.188387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:00.189367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:00.191411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:00.191411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.191411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:00.191411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 318/648] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:00.191411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.205231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.240171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd48712-a59b-4f9b-ae34-b3fff955c270 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.279652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.280382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.280722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.281162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.281778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.282147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.283663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.296808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 331/649] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.296808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.319683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223d21f-2dc1-453a-b9e4-90841b0a1bc8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.345996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.360256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.360885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.361176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.361425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.374883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.376782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:00.396884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:00.398120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:00.399349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:00.399349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.399349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:00.399349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 319/650] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:00.399349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.409129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab9dc-4810-4342-a8e1-20b11e7c8ba3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.445214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.445763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.446129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.446300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.453270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.454370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.465957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:00.467723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:00.468921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:00.469995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:00.473683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 332/651] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:00.475272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.512167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07e6b-1e5c-41e2-be8a-a6fab3ae60c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.549805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.550816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.551298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.551793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.552672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.553192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.554914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.569036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 320/652] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.570068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.583509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9a46-db2f-416c-b89c-deac0796de8d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.595038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.624033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.624653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.624931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.625202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.636573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.638051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:00.654000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:00.655676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:00.656583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:00.656583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:00.656583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:00.656583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 333/653] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:00.656583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.697652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a68ae8-4ba3-442c-a48e-a4eb5737a156 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.722101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.736062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.737222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.737498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.737797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.738501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.738724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.741189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.752256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 321/654] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.753041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.775194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7368-f21a-40c6-8fe5-b5d035a17a72 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.807722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.808322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.808603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.808874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.809569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.809876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.811601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.823183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 334/655] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.823983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.845080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.854622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7584-f529-4b0a-9b3d-7550830aa1fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.888128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.888760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.889049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.889330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.897170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.898651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:00.910767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:00.911708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 322/656] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:00.913127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:00.930177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b066c-0318-44ac-8910-6fc6cf22b0ca None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:00.954987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:00.964369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:00.964969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.965246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.965514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.966156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:00.966491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:00.967864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0c19a-486d-44cb-baca-c6b85ce0bba4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:00.979450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 335/657] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:00.980418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f146f-23a0-40f5-80a1-5dc67a7d642c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:00.980703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f146f-23a0-40f5-80a1-5dc67a7d642c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:00.980971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f146f-23a0-40f5-80a1-5dc67a7d642c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:00.982154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 336/658] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:00.982959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f146f-23a0-40f5-80a1-5dc67a7d642c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.017575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62494d-29cc-4f45-9c00-6b00df751d67 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.051380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.052008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.052289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.052562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.059868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.061556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.062249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:01.073139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:01.074314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 323/659] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:01.075618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.086617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f146f-23a0-40f5-80a1-5dc67a7d642c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.120041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.120641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.120919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.121189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.121906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:01.122209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.123673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.129796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 337/660] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:00 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.130591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.202947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208d80c-41a3-480d-b394-297049762c79 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.206700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.242045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2eed1-8d0c-44d5-8b77-f86dc481ab25 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.247158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.247867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.248214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.248549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.249366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:01.249958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.251703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620f2c-ec9a-44b1-af6d-d7a27b1dec1a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.274473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 324/661] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.274473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.274473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.274473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.285306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.286620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.287009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.287318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.292497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:01.293723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:01.296002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.297558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:01.310182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:01.312116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:30:01.313346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:01.314562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 338/662] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:01.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.320700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.379513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 325/663] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:01.379745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.433905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f2e51-7bc2-46d7-8745-05d92bb5632c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.467476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.468315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.468740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.469158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.470047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:01.470527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.472681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.491804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.492347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 339/664] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.493026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.504728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6f8cf-6738-4d78-a010-8249e179dc83 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.553791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.554423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.554722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.555000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.555744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=25cbe9fdae7b41ad9735c20260ffe22f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:01.556155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.557741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.562464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.565811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 326/665] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/role_assignments?user.id=25cbe9fdae7b41ad9735c20260ffe22f&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.566955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.606803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2eb7f-7291-466f-b12b-dd51a8af6f0d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.645910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.646574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.646945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.647228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.648057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:01.648367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.649803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.668748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 340/666] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.679982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.682644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d9a31-cea6-43e9-ac2e-56c4e9ff186c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.746331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.747076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.747355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.747628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.782477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.820973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac918f3-1176-4bf1-8cb9-d2fc92d42236 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.832519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.833977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.842805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.860497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.861130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.861375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.861706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.944882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.946105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.965943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 327/667] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:01.967578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.058884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.072897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 341/668] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:02.073770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.096590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.102925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccd18-c139-40a0-8bfa-c12f9134c977 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.148323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.150354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.150897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.151374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.165810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:02.173847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.174976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:02.175814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:02.179541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 328/669] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.180675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.185381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd58de-24e5-441f-a8dc-2ae79a4c0544 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.222929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.223589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.223870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.224143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.234941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.237102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:02.257089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.258547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:02.259440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:02.259440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.259440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:02.263245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 342/670] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.263491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.330483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af96b-1223-44f1-921d-ac8e8a3fe903 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.346569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.358262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.379789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.380421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.380706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.380981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.391713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae05b6d-d1a6-4862-ae1e-a8021637903c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.391850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.393725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:02.410119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.411159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:02.412053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:02.412053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.412053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:02.412053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 329/671] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.412734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.442225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.444080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.444630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.445388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.446823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:02.447441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.449300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.465349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 343/672] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:02.467243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.532819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c30177-e54f-4495-8ee0-f29c7abd24fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.570330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.570998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.571276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.571547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.572194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:02.572496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.573919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.586640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 330/673] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:02.587418 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.612490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdf5c8-d744-4906-9fe1-f21542942556 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.623438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.652145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.653073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.653605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.654090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.654974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:02.655480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.657251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.659626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.673004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 344/674] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:02.674817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.712563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa63c96-8cd7-4769-8dfa-2dad3277e1bb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.752034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.752673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.753813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.754107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.766176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.767722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.785118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9c65-73aa-41a8-97bd-11b1655d2af5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:02.785609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.786849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:02.787754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:02.787754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:02.787754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:02.790497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 331/675] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.791525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.833375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.834318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.834685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.834979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.843791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.845656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:02.863350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.864351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:02.865278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:02.869319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 345/676] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.870301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.901180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.925820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bdbee-5977-46a3-951d-3b13162ce981 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.963929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.964632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.964914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.965186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.966818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.973336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.975195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:02.989074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:02.990048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:02.991010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:02.995321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 332/677] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:02.996661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.999696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811b7e-7462-41ba-8fd7-94f848fa9592 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.044350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.045657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.046141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.046885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.048214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.049504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.051850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.068938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 346/678] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.070076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.119700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4d7f-b782-4c19-ae18-15904e018a05 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.181671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.182951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.182951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.182951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.186435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.186435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.186435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.211219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:03.225786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 333/679] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.227177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.229312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a409f6-5de1-48f7-81c3-e04c2502f8ab None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.256206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:03.297119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.297119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.297119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.297119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.298013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.298013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.298561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.321331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 347/680] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.321331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.422679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0a82a-af76-4dea-a005-606e4de8f886 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.489175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.490123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.490407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.490702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.503347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.504916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.516293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1144a-b6c5-4d0d-874e-a6bc0de1d165 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:03.521224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:03.522565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:03.523528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:03.527124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 334/681] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:03.527898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.567031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.567890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.568307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.568725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.569698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=9685e7e865604d4b9acd36902045d141) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.570187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.571795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b885-20a3-403c-86ae-576b1a2030eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.579773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 348/682] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/role_assignments?user.id=9685e7e865604d4b9acd36902045d141&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.581457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73138a-80fc-4fd4-a046-4f3b3a5c501e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.581881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73138a-80fc-4fd4-a046-4f3b3a5c501e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.582281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73138a-80fc-4fd4-a046-4f3b3a5c501e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.583951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 349/683] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:03.584835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73138a-80fc-4fd4-a046-4f3b3a5c501e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.625032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:03.640038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:03.693882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b86fb-d2ec-4703-9cfa-3cc6498af068 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.732177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.733181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.733318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.733723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.734665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=1393d3a8c9174d08b2c6d63cf4e25b19) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.735084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.737357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103e304-5573-452a-bc7d-499d16020bf6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.738505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73138a-80fc-4fd4-a046-4f3b3a5c501e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.749740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 335/684] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/role_assignments?user.id=1393d3a8c9174d08b2c6d63cf4e25b19&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.751877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc298d3-c299-4b92-9abc-d43ee12d97eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.752743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc298d3-c299-4b92-9abc-d43ee12d97eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.753202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc298d3-c299-4b92-9abc-d43ee12d97eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.779892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8cc298d3-c299-4b92-9abc-d43ee12d97eb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:03.780185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.780988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.781380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.781745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:03.782067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8cc298d3-c299-4b92-9abc-d43ee12d97eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:03.782786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:03.783311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:03.785656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:03.805918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 350/685] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:03.812383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:03.872840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 336/686] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:03.937611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:03.948587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d62fb-c2ec-489a-a5a7-24c146d5a715 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:03.991997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:03.992675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:03.992958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:03.993284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.066365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f0b6-2642-425e-986d-48e447d8a417 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.066687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f0b6-2642-425e-986d-48e447d8a417 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.066964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f0b6-2642-425e-986d-48e447d8a417 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.068578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 337/687] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:04.069320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.070991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.073941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.074230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.074530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.095364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:04.095844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:04.160744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.183792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 338/688] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:30:04.193148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.201059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 351/689] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:04.206942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.216272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.329615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81021491-1a4b-4dd8-9ded-d65ff37f9d1c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.362721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3a27d-4706-47b3-bf4a-53e8440be90b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.372526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.372526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.372526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.372526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.390710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.395254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:04.421300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:04.422312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:04.423268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:04.423268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:04.423268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:04.423268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 339/690] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:04.428846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.447005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.448008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.448490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.449020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.467348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.469145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:04.484783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:04.485951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:04.487537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:04.489689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 352/691] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:04.496768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.559296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c944a-fa8b-412d-bd37-c2555e271297 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.582833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.604475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.605097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.605375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.605929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.606600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:04.606913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.609067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.623124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 340/692] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:04.624442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.632467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec581-706a-4522-8368-ef876e96b789 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.682254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.683149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.683441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.683717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.684467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:04.685481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.687198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.702516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 353/693] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:04.703747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.747202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.754971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42199-393f-45ac-a236-65e575b473d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.802272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.803322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.803840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.804289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.817919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.820056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.836770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60158e-ff24-479e-89fe-5199fb1f6012 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:04.838879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:04.839796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:04.840713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:04.840713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:04.840713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:04.842875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 341/694] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:04.843995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.857015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:04.875086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:04.875801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.876149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.876489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.884274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:04.885864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:04.899154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:04.900117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:04.901003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:04.905129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 354/695] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:04.906074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:04.973189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314a4b-7c9b-4980-9101-b0cb09b15ac0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.010860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6d75-8a53-4563-87b9-4bb202c8ca2f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.018744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.020227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.020722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.021176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.036012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.037863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.048237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.048394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.049949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.050241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.050607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.051306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:05.051704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.053304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:05.058919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:05.060134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:05.061092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:05.061092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:05.061092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:05.068322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 342/696] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:05.069536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.070911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 355/697] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.072063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.123158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.191868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c76d-302a-4b35-8ec8-630d001f1ff8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.213619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498ddfc-6818-4a63-8dc2-74b01bb0acca None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.234863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.236030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.236981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.237437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.238844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:05.239470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.241348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.252981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.254059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.254059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.254403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.255207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:05.255599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.257102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.258251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 343/698] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.259550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.276960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 356/699] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.276960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.342600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.380173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.384459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97036-5ad8-4c58-96bf-3ec71740b150 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.401387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eb884-2f3f-4765-a849-c7a839efc6cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.419929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=8daf1537e6fa46238223fd03ced27a43) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.425083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.431773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 357/700] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/role_assignments?user.id=8daf1537e6fa46238223fd03ced27a43&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.432695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.456379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.458002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.458478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.458924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.460082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.462644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.503234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 344/701] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:30:05 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:05.571587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d5ba-f81d-4f11-a0b8-7aaa6a83cb2c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.620490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.621147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.621426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.621697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.635423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.636897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:05.655583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:05.656634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:05.658668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 358/702] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:05.665609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.799304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a0a4-9899-4221-bcff-cd5199e4cf86 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.818356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.839408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.839949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.840229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.840503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.841238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:05.841542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.843080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800194cf-06b9-4731-9f99-9a5a506347de None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.864661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 345/703] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.870133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.905518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.992432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bba88-748b-4746-a94a-da9eca141650 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:06.031679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:06.032585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.033071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.034133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.110822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:06.112654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:06.138180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:06.225814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 359/704] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:06.228926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e429f42-d6ea-434f-b62a-8ab794042423 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.229409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e429f42-d6ea-434f-b62a-8ab794042423 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.229879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e429f42-d6ea-434f-b62a-8ab794042423 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.231402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 346/705] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:06.232576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:06.244132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e60b-4e27-4656-a7bd-949d2f6ecc28 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.249981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e60b-4e27-4656-a7bd-949d2f6ecc28 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.250455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e60b-4e27-4656-a7bd-949d2f6ecc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.307635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ed63e60b-4e27-4656-a7bd-949d2f6ecc28 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:06.308916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ed63e60b-4e27-4656-a7bd-949d2f6ecc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:06.381408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1bf5-10da-49a1-b8e8-77b56b2ba6b0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:06.404300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 347/706] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:06.431661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:06.441459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3313-36b8-4a21-b56d-858a40810374 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.441745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3313-36b8-4a21-b56d-858a40810374 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.442020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3313-36b8-4a21-b56d-858a40810374 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.443683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 348/707] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:06.444991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:06.445628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.445908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.446182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.457841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:06.459352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:06.460415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.460695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.461083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:06.475855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:06.476856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:06.477743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:06.477743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:06.477743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:06.477743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 360/708] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:06.482696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:06.484727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:06.486096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:06.605545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 349/709] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:06.630053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da5941-9c1c-4578-b85d-5b24a5caeff8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:06.665515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:06.666121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.666397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.666689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.667327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:06.667632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:06.668993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:06.681783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 361/710] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:06.687492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:06.726455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:06.730785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:06.792698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:06.896518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a4104-3e75-415b-a42f-e0b907f063d7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:06.897208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e411-7c3d-412c-8b68-9e3f951853f7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:06.936859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:06.937370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.937652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.937919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:06.938538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:06.941088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:06.952580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:06.953642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:06.954511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 362/711] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 15:30:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:06.959794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:06.994899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:06.995880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:06.996358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:06.996827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.004678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.006364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:07.017952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:07.018885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:07.019769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:07.026486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 350/712] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:07.027526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.066627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0866de-f4f3-4385-8419-9ee80bd05d2a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.090878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:07.101554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.102166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.102457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.102939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.103688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.104090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.105727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.111005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 363/713] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:07.117095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.125871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:07.132903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117775b1-5047-499a-bc4c-4489c3d5757b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.178018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.178924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.179322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.179781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] PATH_INFO: `/v3/services/network` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.180422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.181879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:07.194103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:07.195338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:07.196235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:07.201382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 351/714] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:07.202328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.241136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee9ef-ab03-4138-a132-a2ed595d802c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.279058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.279949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.280419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.280864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.281739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.282217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.284351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.297222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 364/715] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:07.298445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.318549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed570d-faa7-4cc2-aef4-6c1bf5110619 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.324648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:07.357801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.358438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.359013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.359529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.360589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.361156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.363232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.377608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 352/716] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:07.377608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.398059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:07.441344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372479d-50a1-46e8-b853-381a1906ce56 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.487649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96643e89-2a3d-401b-b406-ea25eb253e64 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.509789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.510432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.510706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.510976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.511772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] RBAC: Adding query filter params (name=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.511882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.513286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.526221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 365/717] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:07.527537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.528395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.528904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.529377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.530871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.531674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.531847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:07.534634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417909a-8d06-4b8a-bd23-aa1d34608fd5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.541288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 353/718] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:07.696516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:07.714178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aed6e-ac70-4194-b73d-1734b96ac523 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:07.750833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:07.751359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.751640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.751952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.752581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] RBAC: Adding query filter params (type=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:07.752886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:07.754251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb9a0f-c332-4b98-8f78-76630d4b3d74 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:07.760511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 366/719] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:08.310123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd03d-7983-432b-a368-0c6667ca3afa None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.311122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd03d-7983-432b-a368-0c6667ca3afa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.311591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd03d-7983-432b-a368-0c6667ca3afa None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.313375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 354/720] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:08.319722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.320301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.320775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.339435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:08.341009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:08.377273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589ee0-352e-4b46-ba53-bc2dfa268df8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.377657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589ee0-352e-4b46-ba53-bc2dfa268df8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.377828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589ee0-352e-4b46-ba53-bc2dfa268df8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.379651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 355/721] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:08.385570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.385865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.386156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.407629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:08.408883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:08.444625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 367/722] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:30:08.486612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 356/723] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:08.787944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.851696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.874340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:09.004438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:09.046935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7bac6-0885-4d8d-9ba7-51c3d6434819 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.050705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d8b26-5043-4659-ab28-4c9606cea806 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.118259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.120205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.121547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.122022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.126461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.128428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.129302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.131644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.144214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:09.166012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:09.167428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:09.168361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:09.169219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:09.175396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:09.176482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:09.177352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:09.179840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 357/724] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:09.186720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 368/725] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:09.194121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.202876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.356728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:09.401990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e27f-ab96-467f-ae94-42a42f8c50c1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.446044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4b9de-25bc-4807-b463-3d68851dc0d6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.479797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.480761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.482209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.482718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.485026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:09.485573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:09.488949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:09.514368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 358/726] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:09.515703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.525045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.525983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.531703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.532257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.533720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:09.534313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:09.536035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:09.560872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 369/727] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:09.572900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.577084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:09.701145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18cbf3-8a4b-4943-ac3d-b305257eaf92 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.739927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287ffb3-baea-4167-bbb1-276ba731f9c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:09.740753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.741648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.742183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.742741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.755344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:09.757111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:09.772904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:09.773908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:09.775215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:09.775215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:09.775215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:09.776992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 370/728] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:09.782842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.844355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:09.845019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:09.845978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:09.846276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:09.866786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:09.871858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:09.873370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:09.919341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:09.920347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:09.921252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:09.921252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:09.921252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:09.921252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 359/729] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:09.926690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:09.997916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.078189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bb16-6faf-4297-aa4f-c12181526961 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.101270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545491-086a-4bdf-b1b6-7fe32acbcef2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.147307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.147929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.148212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.148486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.149134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:10.149438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.152263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.171332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.171954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.172238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.172509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.173158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:10.174079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.175488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.185698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 360/730] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:10.189584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.214856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 371/731] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:10.221313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.290935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.365693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d7a-d7aa-40e0-9a7d-e38ca3f84764 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.403288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cab058-087d-41d9-a07d-1dcb5a7d0010 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.406841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.407487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.407799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.408074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.416226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.417712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.421993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f736d205-0605-48b3-bee8-3532eb13ce0a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:10.431343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:10.432310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:10.433271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 361/732] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:10.433848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218582-de82-40cd-909a-3042dedced81 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.434490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218582-de82-40cd-909a-3042dedced81 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.434490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218582-de82-40cd-909a-3042dedced81 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.438652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 362/733] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:10.438652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e749f1-6f61-4b0b-b581-85f0a71c74b6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.438652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e749f1-6f61-4b0b-b581-85f0a71c74b6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.438652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e749f1-6f61-4b0b-b581-85f0a71c74b6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.442504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 363/734] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:10.442504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e749f1-6f61-4b0b-b581-85f0a71c74b6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.459051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.459750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.460000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.460227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.467775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.469338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e393eda4-5722-49c9-b216-d3662d40269e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:10.481731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:10.482747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:10.483676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 372/735] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:10.484748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a8b2-2ef4-47cc-953c-3103a6b56224 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.484748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a8b2-2ef4-47cc-953c-3103a6b56224 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.484986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a8b2-2ef4-47cc-953c-3103a6b56224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.504334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-66f1a8b2-2ef4-47cc-953c-3103a6b56224 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:10.505621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-66f1a8b2-2ef4-47cc-953c-3103a6b56224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:10.554676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e749f1-6f61-4b0b-b581-85f0a71c74b6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.566815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.576268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 373/736] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:10.577700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f34f-5d79-46ec-952d-642af9129639 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.577989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f34f-5d79-46ec-952d-642af9129639 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.578287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f34f-5d79-46ec-952d-642af9129639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.597324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9be5f34f-5d79-46ec-952d-642af9129639 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:10.597843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.598564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.598688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9be5f34f-5d79-46ec-952d-642af9129639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:10.598906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.599246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.600055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:10.600431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.601852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.620104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 364/737] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:10.620981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.675249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 374/738] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:10.677029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa4a3-4c9a-450a-8879-1905c9cc56b1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.677523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa4a3-4c9a-450a-8879-1905c9cc56b1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.678028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa4a3-4c9a-450a-8879-1905c9cc56b1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.679818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 375/739] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:10.680777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa4a3-4c9a-450a-8879-1905c9cc56b1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.737517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f48bc9-cbe2-45fb-a766-bad9d4988840 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.739175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.783164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.783777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.784128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.784406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.785139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:10.785441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.786891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512ec07-2c2e-4f66-9006-c70da20b1006 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:10.811963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 365/740] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:10.813514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.813805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.814643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.832069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa4a3-4c9a-450a-8879-1905c9cc56b1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:10.840205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:10.841404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:10.867936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:10.870406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:10.871051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:10.871378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:10.871676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:10.926889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 366/741] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:30:10.926889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:10.959043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:10.960723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.000421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.043617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db40ddd-1650-45e6-93eb-b668dbfb7155 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.067423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 376/742] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:11.072366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.080103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.080709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.080994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.081266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.113933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.158927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.160463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.176793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f8f5-bd83-47e7-aac3-21b32bc38010 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.214660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.215276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.215555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.215827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.216488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.217899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.222087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.234774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 377/743] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:11 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:11.235630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.279198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 367/744] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:30:11.279870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.346993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.375616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7890-e5c5-40a0-809b-15cac61beb09 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.397107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185041-0e21-4f22-bddc-e973383512c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.420252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.421149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.421436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.421718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.433913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:11.439788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:11.440718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:11.441552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:11.442188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.442289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 378/745] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:11.442362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.442462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.442537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.442537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.455854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.457693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:11.474400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:11.475425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:11.476318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:11.476318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:11.476318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:11.476318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 368/746] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:11.476563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.506225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.580126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514f44-0bae-4ba9-b9b5-e34cc3ff30a7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.598100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c5ea-5793-4889-a1c1-bbcea45dd2b7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.607131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.619033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.619638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.619912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.620178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.620824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.622178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.639216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 379/747] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:11 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:11.640050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.672592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.674271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.674875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.675304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.691765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.694137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:11.713721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:11.715031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:11.716203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:11.716203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:11.716203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:11.717118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 369/748] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:11.718142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.778535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355d030-65fd-4632-9a38-75e9cf4502c2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.791814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.832455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.833402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.833908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.834858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.835120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:11.835510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.837126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.855841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 380/749] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:11.857288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.882404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b85af-6631-4dcd-b83d-88457e672eb4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.923236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.924504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.924843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.925186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.925888 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:11.926253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.927722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.943087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 370/750] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:11.944182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.971068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.980727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef94-b482-4f39-8f25-0c2e0c41c9a6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.015515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.016483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.016957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.017965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.018860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.019388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.020958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.033606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 381/751] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.034929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.080880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5945557-0816-494f-9d62-7c86d6d2cd7c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.094860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.120453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.121407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.121911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.122428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.132871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.134719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:12.152273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:12.153288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:12.154202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:12.154202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:12.154202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:12.161356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 371/752] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:12.162730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.171659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c2d7-ba16-4c47-9936-ffbea64cfde9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.219925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.221491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.221782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.222062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.230720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.232773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:12.247815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:12.248742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:12.250270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 382/753] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:12.250785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.262816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.289550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703416e-d027-41d0-8e98-dae7ea814d3b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.325965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.327138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.327721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.328195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.335814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.337769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:12.350604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:12.351522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:12.352423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:12.353893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.356434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 372/754] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:12.357553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.365559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfdb93-ddde-45bd-a86f-bdc34f79e274 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.401199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.401829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.402100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.402367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.403060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.403366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.404742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.418907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 383/755] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.418907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.483037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe74c-9a13-4ef5-ac36-365697561a51 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.518486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.523918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449ea0-e4bf-44f0-9e27-b920196aa710 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.535634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.536690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.537166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.537651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.538638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.539143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.540889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.556744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 373/756] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.557999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.572063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.572697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.573210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.573341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.574016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.574329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.575886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.594354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 384/757] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.596188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.623105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.690634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a7eb-e64f-47b6-9d5e-d1ec1ba64733 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.728684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28049459-557d-4a6e-9163-b6f50631ef6a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.729130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.729817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.730099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.730369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.738829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.740332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:12.751994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:12.752955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:12.753887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:12.757817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 374/758] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:12.758655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.769740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.770649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.771124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.771605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.772517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=d5846f447c204206bde8acaa4160664e) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.773016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.774646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.780638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 385/759] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/role_assignments?user.id=d5846f447c204206bde8acaa4160664e&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.781707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:12.812311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.869448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:12.877804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb420-dc97-458c-97b8-6cbf136564a5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.918624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.919404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.919623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.919863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.920922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=8daf1537e6fa46238223fd03ced27a43) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.921460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.928367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.929198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5ed4-d6c2-4145-b2eb-37bb75c3f0cb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:12.951081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 375/760] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/role_assignments?user.id=8daf1537e6fa46238223fd03ced27a43&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:12.973520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:12.974555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:12.975046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:12.975537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:12.976496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:12.977023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:12.978750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:12.999682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 386/761] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:30:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:13.006321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:13.212092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9fbb-4f10-4336-b8e2-d7c3bd5c91bf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:13.234554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:13.296983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:13.297634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.297914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.298247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.315788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:13.317249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:13.340521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:13.341519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:13.342858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:13.342858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:13.342858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:13.342858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 376/762] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:13.351968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:13.551012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff926-fe12-4c46-af12-fd8503f5e121 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:13.592010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:13.592631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.592908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.593178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.593831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:13.594137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:13.596196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa0bc9-fa2a-458e-af5c-e0843f02af64 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:13.599062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:13.615561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 387/763] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:13.621040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:13.762621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59f3db-4b8a-4391-8fba-75ebaf3b797b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.762910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59f3db-4b8a-4391-8fba-75ebaf3b797b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.763187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59f3db-4b8a-4391-8fba-75ebaf3b797b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.764651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 388/764] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:13.770062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401528e5-6084-40c2-a706-9db279086bbe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.770352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401528e5-6084-40c2-a706-9db279086bbe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.770664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401528e5-6084-40c2-a706-9db279086bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.804653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-401528e5-6084-40c2-a706-9db279086bbe None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:13.806969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-401528e5-6084-40c2-a706-9db279086bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:13.816941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2605f0-2ff6-4a58-9e49-3ea727eebc34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:13.979827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:13.980922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.981417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.981892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.014443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:14.014443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:14.053248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:14.054197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:14.055197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 377/765] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:14.061394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.118440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 389/766] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:14.158761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:14.381214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cf3ae-d141-430c-8fb7-89b4a71d140c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.381516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cf3ae-d141-430c-8fb7-89b4a71d140c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.381685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cf3ae-d141-430c-8fb7-89b4a71d140c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.382328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca56f-65c9-4680-a737-8c8828f16ea3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:14.383366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 390/767] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:14.389658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.389792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.390039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.452495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:14.466302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:14.503128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:14.503747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.504034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.504313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.505028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:14.505370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:14.515349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:14.543554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 378/768] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:14.545553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.648915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:14.669005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70ce2-0f63-44c1-a818-736117c66631 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:14.710844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:14.712238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.713317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.714003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.722435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:14.724131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:14.724590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 391/769] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:14.725970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:14.756797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:14.757874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:14.758787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:14.772243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 379/770] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 15:30:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:14.795459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.908770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:15.065306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f23fb-bfbc-455d-849f-a31651d5bb67 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:15.165182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:15.165783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.166062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.166329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.168251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=6fcc9eec4e114f538c506178c744b49a, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:15.169699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:15.212568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 392/771] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:14 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:15.213268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:15.218445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef26b12-1be8-44e3-b3d8-5c4ddeef789b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:15.291899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:15.291899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.292335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.292501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.293352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:15.293476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:15.307073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:15.332604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 380/772] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 537 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:15.343782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:15.411053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:15.532969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58176-3231-4dfc-ac78-be61b5e4a431 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:15.553389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:15.579757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d549e-67db-42cc-8f42-918e1d937062 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:15.589338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:15.590218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.590649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.591030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] PATH_INFO: `/v3/services/network` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.591933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:15.594014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5ecf00bd-ff93-4d56-9998-7864a659577b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:15.609328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:15.610357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:15.611879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 393/773] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:15 2022] GET /identity/v3/services/network => generated 88 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:15.613317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b791b4-1be0-45f5-8db4-72d45074c2b7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.613601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b791b4-1be0-45f5-8db4-72d45074c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.613879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b791b4-1be0-45f5-8db4-72d45074c2b7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.620278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 394/774] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:15.620278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b4a6-3556-4726-a444-1797b589a538 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.620278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b4a6-3556-4726-a444-1797b589a538 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.620278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b4a6-3556-4726-a444-1797b589a538 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.623891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 395/775] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:15.623891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216b4a6-3556-4726-a444-1797b589a538 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:15.635218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:15.636857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.637685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.638357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.640764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:15.640764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:15.642999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e33a-c6fa-4b26-996a-1580f1cf6145 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:15.655027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 381/776] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:15.658262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efab33-808a-45cf-baeb-457d2d10a93c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.659169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efab33-808a-45cf-baeb-457d2d10a93c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.659984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efab33-808a-45cf-baeb-457d2d10a93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.682776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b6efab33-808a-45cf-baeb-457d2d10a93c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:15.684246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b6efab33-808a-45cf-baeb-457d2d10a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:15.756584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216b4a6-3556-4726-a444-1797b589a538 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:15.757775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 382/777] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:15.759248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.759529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.759803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.773491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:15.784757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:15.785889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:15.836965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:15.838341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:15.839222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:15.839844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:15.842153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] RBAC: Adding query filter params (name=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:15.843289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:15.846816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:15.860193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 396/778] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:15.860193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:15.873589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 383/779] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:15.874593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:15.936969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.055272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855c7d4-4769-4abc-9865-e574fec34d61 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.080070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641195c8-3d07-47db-96ac-e05555d57858 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.106520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.108144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.108795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.109393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.110718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] RBAC: Adding query filter params (interface=public, service_id=58d14d8297304491958ab9265527b5e9, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:16.111648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.113475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.123345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.124001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.124223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.124592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.124801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 397/780] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:15 2022] GET /identity/v3/endpoints?service_id=58d14d8297304491958ab9265527b5e9&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.125184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] RBAC: Adding query filter params (type=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:16.125488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.126825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.127435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.130275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.139952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 384/781] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.139952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.248356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476fdef-ff69-4e8f-99a7-c9b4cc915711 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.255397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f43ed-9a52-4484-9b39-1cb838a37285 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.256694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.286081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c24555-00e7-4076-a732-574221224e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.286755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24555-00e7-4076-a732-574221224e3a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.287029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24555-00e7-4076-a732-574221224e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.287302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24555-00e7-4076-a732-574221224e3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.296797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.297492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.297761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c24555-00e7-4076-a732-574221224e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:16.297860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.298187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.298886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.300319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-01c24555-00e7-4076-a732-574221224e3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:16.302639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:16.303529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:16.304366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 398/782] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:16.305061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c24555-00e7-4076-a732-574221224e3a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.319464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 385/783] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.319464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.407176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.436104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c24555-00e7-4076-a732-574221224e3a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.462800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93d32-425a-4fe7-b891-d151954f3fd5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.479290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.480054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.480449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.480812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.481735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] RBAC: Adding query filter params (interface=public, service_id=792cc3610e0e4b3b8b6cf20e9a824596, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:16.482135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.483737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.490779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 399/784] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/endpoints?service_id=792cc3610e0e4b3b8b6cf20e9a824596&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.491855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.510983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.511620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.511902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.512171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.525583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:16.527710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:16.531360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:16.532416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:16.533748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 386/785] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:16.535211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.610270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f647c-ee26-449a-af2a-fe7246e29dbc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.644916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.645519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.645796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.646065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.646803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:16.647066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.648471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.661875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 400/786] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.666940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.678970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5059c6-7faf-4dac-ae9a-e4f07d31785d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.712622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.728357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.729448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.729905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.730589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.731769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.733815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.748265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 387/787] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.748265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.798606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:16.814891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24426791-6c39-4d87-90db-2f647e99f182 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:16.881736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:16.883177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.886438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.886438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.886438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:16.886438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:16.887649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:16.908484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 401/788] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:16.914006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:16.982099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52850-7943-41fe-8561-5764671bb3f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.029476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.030582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.031065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.031719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.046271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.048849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.053729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3bf8-cdf5-46df-b6e0-08b7c847d68a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:17.068639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:17.069849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:17.070831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:17.070831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:17.070831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:17.071950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 388/789] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:17.077563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.094936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.095646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.095893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.096696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.116225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.117843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.136933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:17.137930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:17.138879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:17.139810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:17.139810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:17.139810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:17.139810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 402/790] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:17.147555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.160954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.208935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fbeeb-1ab4-4117-a7b6-2433140b2b9c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.246589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.247570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.248072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.248547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.249439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:17.249939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.252281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.268665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 389/791] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:17.276832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.383217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36122ef9-ca29-499c-b4f5-0f962605b9af None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.425233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.426280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.426778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.427304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.428606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:17.429108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.430709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.444638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 403/792] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:17.451579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.494801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.499282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a5c58-4d93-4318-b632-9fa0302c2cec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.537372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.538012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.538292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.538601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.546733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.548839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f3d51f59-9838-4620-9fe8-e6f0f58c49c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:17.568773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:17.569760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:17.570726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:17.571276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.571351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 390/793] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:17.573417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d256b-f9e4-441a-b3e4-5bc354291665 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.573696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d256b-f9e4-441a-b3e4-5bc354291665 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.573972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d256b-f9e4-441a-b3e4-5bc354291665 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.578439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 391/794] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:17.579579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d256b-f9e4-441a-b3e4-5bc354291665 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.622689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661b1d-ae52-4558-8538-354081cb970d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.661982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.661982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.662355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.662665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.674899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.676550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6bc6f0f9-2e70-4e3c-91dd-22ff569a3945 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:17.691295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:17.693235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:17.694922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:17.700420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 404/795] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:17.700420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.700420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.700420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.718354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:17.719213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d256b-f9e4-441a-b3e4-5bc354291665 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.719721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:17.766336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.768468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.774436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.774436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.774436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:17.774436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.774436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.798755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 392/796] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:17.799644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.803111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 405/797] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:17.804031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.842109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.894438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.983656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a147-d604-4937-b1df-f8e576a1cd43 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.012378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3efee-a1f0-40ad-8380-193e2b668a0b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.056417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.057048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.057329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.057600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.080531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.082006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.083914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.084803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.085215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.085566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.086310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.088184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.089634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:18.117710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:18.120223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:18.122272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:18.122272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:18.122272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:18.122272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 406/798] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:18.127764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 393/799] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:18.129737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.138437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.224670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.246618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ae2c7-6e28-4480-b93c-6f3e473eadd2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.254446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718c32-614f-40e7-90ec-59f9cfbd4330 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.259042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.291423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.292308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.292595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.292870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.293632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.293941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.295511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.297056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.297921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.298397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.298844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.310356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.310801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 407/800] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:18.311718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.312631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:18.330856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:18.331861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:18.332758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:18.332758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:18.332758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:18.339537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 394/801] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:18.340812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.445301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68173b6a-2756-42e6-9136-9295209df3af None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.456266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9367d2-8be1-43a2-a768-5b7be2d5b3f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.491159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.491784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.492066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.492341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.494062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.505546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.505709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:18.506602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.507084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.507552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:18.510396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:18.511342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 408/802] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:18.517528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.524373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.525847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:18.538387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:18.539359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 395/803] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:18.540857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.556749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.655937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3075e-52db-4aac-ab56-6e42296b83e5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.669116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2720d2-cb87-41dd-8bdf-528216575a76 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.702449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.703690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.704740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.705464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.707012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.707535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.710044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.713952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.714654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.714940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.715271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.715942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.716251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.717697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502dc11-d1ac-456d-91b9-064c5f77a02a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.724960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 409/804] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:18.726478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.734248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 396/805] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:18.734248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaeb24-0755-4dc5-a2f8-d4410d123012 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.734248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaeb24-0755-4dc5-a2f8-d4410d123012 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.734248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaeb24-0755-4dc5-a2f8-d4410d123012 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.734248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 397/806] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:18.736490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaeb24-0755-4dc5-a2f8-d4410d123012 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.791973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.821308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:18.835252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc9bb2-23e3-41e4-805c-b325197424b8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.852095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaeb24-0755-4dc5-a2f8-d4410d123012 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.874455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.875140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.875431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.875680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.876918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.877220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.878772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.889736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.890374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.890681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.891016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.896442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 410/807] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:18.896442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.906161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.907720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bd69d70b-cdcb-45aa-a576-314d63ff670d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:18.920779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:18.921705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:18.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 398/808] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:18.923718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.923796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.924375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.942333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:18.943539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:19.015570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 399/809] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:19.016313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.017562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8aa24-36fc-4545-9449-d68c15d51936 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.065559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.071921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.072456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.072734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.073088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.084033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.085476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.093829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:19.101828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:19.102830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:19.104332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:19.104332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:19.104332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:19.104332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 411/810] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:19.104332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.135500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bd9f6-5d89-42c7-8f34-b328b2c6fd44 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.175975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.177018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.177523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.178012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/d5846f447c204206bde8acaa4160664e/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.193921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=d5846f447c204206bde8acaa4160664e, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.195734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.213905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 400/811] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:19 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/d5846f447c204206bde8acaa4160664e/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:19.215480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.222034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811d479-2f6d-4cd5-b406-e93a34b3e5c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.271161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.271534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.271863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.272137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.272881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:19.273187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.274746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.288440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 412/812] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:19.294357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.341910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.360889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499291e3-a1aa-41fd-8c66-56e26825663b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.367952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.399238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.399860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.400141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.400399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.401053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:19.401355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.402800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.414940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27538997-c155-4c42-85db-c89b07ea1326 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.419776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 401/813] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:19.420825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.459384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.460015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.460292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.460565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.461152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.463146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bcd38835-42ea-4a1b-832b-be1e5c2a6ae1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:19.472598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:19.473589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:19.474481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 413/814] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:19.475952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21b6c-16b0-4e0b-88b6-988ceade2948 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.475952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21b6c-16b0-4e0b-88b6-988ceade2948 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.475952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21b6c-16b0-4e0b-88b6-988ceade2948 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.476814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 414/815] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:19.477309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd21b6c-16b0-4e0b-88b6-988ceade2948 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.538905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f60f51-9789-43a7-bafa-67dd27751a42 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.578782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.579609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.579884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.580154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/8daf1537e6fa46238223fd03ced27a43/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.582321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=8daf1537e6fa46238223fd03ced27a43, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.584189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.590405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd21b6c-16b0-4e0b-88b6-988ceade2948 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.599702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 402/816] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:19 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/8daf1537e6fa46238223fd03ced27a43/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:19.601128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.623983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.627584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.632447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.633421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.635584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.635861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.650501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.652493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-72f7e291-7282-43a4-9e3f-280bc89599de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:19.667833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:19.669049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:19.670033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:19.673698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 415/817] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:19.675389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.675941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.676487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.701416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:19.703172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:19.764711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cae78-0896-4064-ba2a-421736814c24 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.802441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 416/818] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:19.803539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.815701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.816323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.816603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.816875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.817520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:19.817825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.819251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.829848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 403/819] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:19.835795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.915582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1005e3-21fc-4b68-b013-e5fc8ecca414 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.919976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.933211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:19.947537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f22ec-e629-49c0-9146-14a35e7243d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:19.956395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.957206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.957686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.958099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.959062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:19.959510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.961079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.982405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 417/820] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:19.985869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:19.986587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:19.986940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:19.987527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:19.988171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:19.988479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:19.988904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:19.989960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:19.995671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 404/821] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:20.001301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.098380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564342b3-1941-4f2b-8ade-2fc2654c3275 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.147772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.149702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.151073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.152883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.160551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d357d-7942-4cac-892f-50b0e9ffc36d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.217071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.218057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.218556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.219022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.220493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.222182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.223947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.238241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.265706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 405/822] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:20.274519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.279028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.280823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.436187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1193a3-caef-49ba-80b8-d2a4365641db None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.483858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.484439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.484723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.485003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] PATH_INFO: `/v3/services/network` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.485799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.487284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:20.498621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:20.499674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:20.500518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 406/823] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:20.505542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.524369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 418/824] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:30:20.530172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.573619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.627975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1195e-1f77-432f-a100-a14088ca00ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.628762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.674847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c67a5-099f-4534-8f07-42f8d11ef93b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.679613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.680565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.681051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.681525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.682603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:20.683155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.687244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.693371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 407/825] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:20.703527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.737527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.737527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.737527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.737527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.756605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.756605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:20.786532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:20.787518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:20.788416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:20.788416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:20.788416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:20.788416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 419/826] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:20.799397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.907016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e907-9f91-43c0-8e33-1e52513195ac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.946772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.980592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.981209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.981485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.981755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.982385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:20.982708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.984075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.001672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 408/827] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:21.004751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.039808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae3c-630f-45d3-934b-dd01b7942598 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.087773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.105934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.107424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.107706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.107979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.108628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:21.108931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.110363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.138470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 420/828] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:21.146444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.217438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f122378-89e2-4e70-8c25-fea3503450d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.316079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4854f-35c9-46ed-8220-ff85ade14dea None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.323144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.323769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.324049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.324319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.324996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.326360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fbe5-a887-4fde-b28b-ce1e0f126844 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.352058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.352687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.352964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.353234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.360829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.362283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:21.385044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:21.387029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:21.388918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:21.394455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 421/829] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:21.394455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.406599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 409/830] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:21.492915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.589509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80435ee9-dd88-4170-9e0d-a22e55bcc445 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.636229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.636579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520e38-10dc-419f-b955-0b0a795e1726 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.636835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.637032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520e38-10dc-419f-b955-0b0a795e1726 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.637192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520e38-10dc-419f-b955-0b0a795e1726 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.637341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.637341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.638047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:21.638349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.639887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.641632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 410/831] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:21.645064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.645351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.645632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.654546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 422/832] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:21.667956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.670440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:21.671876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:21.709193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.766964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 411/833] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:21.792013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174f19-7d28-4fe1-bd25-936e8869900f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.796582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.853245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.853784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.854065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.854339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.855162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:21.855474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.856911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.864557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 423/834] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:21 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:22.077483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.186454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77871-1563-4809-bdcc-b47484889a39 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.219696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:22.220337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.220618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.220893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.231513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8bfe431f-8482-415c-9046-bb4fd3d8f18f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:22.236511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:22.238106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:22.241216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 412/835] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:22.247173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.385070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25fb2f-a789-4919-b76a-9ecf021a7c5a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.393825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3634-bb2e-47fe-8aaa-82027f1d9323 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.393825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3634-bb2e-47fe-8aaa-82027f1d9323 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.393825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3634-bb2e-47fe-8aaa-82027f1d9323 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.397245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 413/836] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:22.402964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.403257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.403534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.424082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:22.425330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:22.430325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:22.431228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.431710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.432202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.433062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:22.433679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:22.435533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:22.446448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 424/837] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:22.456038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.459119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:22.506794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:22.528940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 414/838] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:22.591597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4e38e-c8b4-4971-b0fc-6eac321febd2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.627902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:22.628536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.628882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.629158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.640690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:22.642810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:22.658470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:22.659460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:22.660539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:22.660539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:22.660539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:22.660539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 425/839] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:22.667869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.796093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d714101-f426-4808-a61d-6267ab76c4aa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.798978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.836644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:22.842434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:22.854491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 415/840] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:22.859492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:22.882842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:22.900151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:22.926435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd60a8a-4201-4d3e-a740-f31f8f305218 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.965630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:22.966255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.966576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.966852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.976739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9596f-d2d4-4943-b51b-56111b2d576a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:22.977754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a06925e3-9a78-489d-8c77-9679f6e23611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:22.984450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:22.985970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:22.986971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 426/841] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:22.988208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e14a-d1e6-4a2d-9881-42ad21f57953 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:22.988208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e14a-d1e6-4a2d-9881-42ad21f57953 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:22.988497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e14a-d1e6-4a2d-9881-42ad21f57953 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:22.993664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 427/842] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:22.993664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e14a-d1e6-4a2d-9881-42ad21f57953 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.020738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.021378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.021668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.021929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.030568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.032559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5ef32573-7bff-49c4-9dcd-b1bd2f23b7f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:23.043845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.044973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:23.045940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 416/843] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:23.046620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.046620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.046755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.068291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:23.069486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:23.103586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e14a-d1e6-4a2d-9881-42ad21f57953 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.137453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 417/844] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:23.137972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.138160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.138590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.138869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.139545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:23.139853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.141225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.146625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.147415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.154791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 428/845] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:23.159774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.164487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.282530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdd257-2a51-468b-a52f-12f07a1a9eda None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.302125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b59de-2e24-4295-91df-05b8e6dd9a2c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.318073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.318859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.319233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.319603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.320434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:23.320839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.322484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.341110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 418/846] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:23.346530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.346920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.347593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.347970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.348470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.361624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.363273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:23.383324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.384587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:23.385666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:23.385666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:23.385666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.385666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 429/847] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:23.391442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.429325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.452510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.469229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a775b-30d1-4670-9ce8-f9d49f736040 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.508416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.509358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.509641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.509916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.511171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370d23a-1202-456d-9bf7-8377b2419998 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.548801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.549410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.549707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.549964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.550634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:23.550942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.552382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.565143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 430/848] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:23.567290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.585487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.586981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.654394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.679898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780a1e-3425-4b35-8285-355bdcdee141 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.693888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.699368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 419/849] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:30:23.699368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.726514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.727346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.728040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.728325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.729190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.731239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.743234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.744225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:23.745068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.746684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 431/850] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:23.749162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.806220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af960ba-b09c-477f-ba21-c15fcc46d784 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.846736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.847471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.847827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.848167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.856660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.858201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.861671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2eaaa-90ce-471f-bc84-1fe81033fbff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.899998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.900649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.900926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.901196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.916499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:23.918029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:23.936706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.937738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:23.938833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:23.938833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:23.938833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.938833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 432/851] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:23.939568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.971180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:23.989771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.990894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 420/852] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:23.991873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.021493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.058837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962be6a-42b3-4e4b-9127-df971372009f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.095627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.096259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.096543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.096816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.097473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.097786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.099260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.104508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 433/853] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.105374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.115078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b80b0-16f5-4e3b-8890-bea157a0d522 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.155773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.157591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.158019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.158500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.159145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.159449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.160810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.177444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 421/854] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.177444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.213875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.222988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cad8d-ccd1-4985-b217-6b2a43b9c080 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.263807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.264417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.264698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.264977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.265698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.266007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.267015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.267494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.281404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b766-9116-4de2-b532-29c524187b2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.284825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 434/855] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.285705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.316511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.317049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.317329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.317600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.318226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.318562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.319916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2839b6-ac10-470f-8faa-a64d69993fc8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.324150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 422/856] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.325286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afebe19-4951-4467-86b4-125ba84005a1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.325580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afebe19-4951-4467-86b4-125ba84005a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.325839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afebe19-4951-4467-86b4-125ba84005a1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.327083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 423/857] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:24.327803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afebe19-4951-4467-86b4-125ba84005a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.404578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668279-1afa-4bdb-8e1f-e50fbc6a9c6a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.441896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.442684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.443029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.443369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.450908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.452497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.457610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afebe19-4951-4467-86b4-125ba84005a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-157330a8-bb3d-4e59-910b-558437c50596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:24.465584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:24.466598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:24.467496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:24.472851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 435/858] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:24.474305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.474698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.475169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.495646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:24.497096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:24.498962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.499605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.499881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.500156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.501012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.524188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.588492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.589112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 436/859] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:24.589986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.590904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.728652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 424/860] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:24.729634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.734156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2df1-b21d-4ef9-ac73-1e4ca07833a2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.780423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.781177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.781458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.781731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.782598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.783390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.783390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.785104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.796387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.806647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 437/861] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.808355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.846904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b855af-66eb-41e3-9ed1-a46511f02ab2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.882390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.883045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.883675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.883949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.898893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.900446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:24.920750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:24.921854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:24.922912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:24.922912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:24.922912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:24.923773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 425/862] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:24.925210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.935237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362bbd2-9c9a-4016-b2b2-914e1d4c2d67 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.968782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.970018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.970361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.970743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.971527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=d5846f447c204206bde8acaa4160664e) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.971898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.973395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.986611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 438/863] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/role_assignments?user.id=d5846f447c204206bde8acaa4160664e&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:24.987728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.054117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29375a3f-6cd9-4aa7-b3ad-d7bfc5f49503 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.056568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:25.103453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.104021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.104299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.104570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.105450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.107069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.115001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:25.120248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:25.121327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:25.122242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:25.123845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 426/864] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:25.129982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.179175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f75f5-4a61-441a-8f7d-74455bf15446 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.242628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.243730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.244010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.244288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.244950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:25.247042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.247042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.263152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 439/865] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:25.267456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.284858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bd14-250d-48ff-aed2-0075ca860e39 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.322075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.322720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.323046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.323342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.323970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:25.324268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.325623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.331216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 427/866] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:25.336947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.379391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:25.409540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0af1-776f-4c64-a273-f77a1a0b52be None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.444298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:25.462857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.464064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.464783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.465645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.472591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0698b5-acb7-47f3-aae8-2c9e938d2d4a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.476115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.477803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:25.495339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:25.496313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:25.497200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:25.499750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 440/867] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:25.505371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.515853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.517026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.517309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.517580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.518204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:25.518555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.519915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.525012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 428/868] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:25.630684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17da4a5-8cec-40ea-adfc-328d904b67cd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.669016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.670058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.670578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.671105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.672020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:25.672684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.675218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0a45-3ca5-4cd5-b4d5-4c3d4b06c504 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.689943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 441/869] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:25.696346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:25.804387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:25.811778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dc5c7-eb70-4065-8f93-304bbe6a0c8b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:25.847063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:25.847816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:25.848186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:25.848548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:25.849372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=8daf1537e6fa46238223fd03ced27a43) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:25.849774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:25.851323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc3259-fb5c-4996-ac6a-078f3ad2da30 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:25.862804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 429/870] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:25 2022] GET /identity/v3/role_assignments?user.id=8daf1537e6fa46238223fd03ced27a43&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:26.575347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620d2c-84e9-4161-ae17-6492646f012f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:26.575905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620d2c-84e9-4161-ae17-6492646f012f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:26.575905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620d2c-84e9-4161-ae17-6492646f012f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:26.577372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 442/871] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:26.582941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2b87-153c-480c-bf75-17a8091bd978 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:26.583361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2b87-153c-480c-bf75-17a8091bd978 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:26.583470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2b87-153c-480c-bf75-17a8091bd978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:26.602048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f50a2b87-153c-480c-bf75-17a8091bd978 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:26.603238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f50a2b87-153c-480c-bf75-17a8091bd978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:26.681606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 430/872] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:26.956791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620d2c-84e9-4161-ae17-6492646f012f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.007840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:27.065260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620d2c-84e9-4161-ae17-6492646f012f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:27.101347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:27.101949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.102232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.102525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.103185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:27.104531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec622cf-f3be-442d-8996-2392cee9e54b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:27.121369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 443/873] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:26 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:27.382196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f07800-aa93-4dfb-88e6-de8b27fce791 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.382675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f07800-aa93-4dfb-88e6-de8b27fce791 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.383057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f07800-aa93-4dfb-88e6-de8b27fce791 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.384774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 431/874] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:27.390290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a68c-044f-46cd-a948-487ad865a9c6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.390618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a68c-044f-46cd-a948-487ad865a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.390894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a68c-044f-46cd-a948-487ad865a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.409964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d9a0a68c-044f-46cd-a948-487ad865a9c6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:27.411204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d9a0a68c-044f-46cd-a948-487ad865a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:27.492248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 444/875] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:27.768978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f07800-aa93-4dfb-88e6-de8b27fce791 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.878309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:27.884887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f07800-aa93-4dfb-88e6-de8b27fce791 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:27.895532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1dccd-23b1-4010-9bd8-34dc334dc96e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.896530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1dccd-23b1-4010-9bd8-34dc334dc96e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.897020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1dccd-23b1-4010-9bd8-34dc334dc96e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.903079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 445/876] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:27.904875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.905372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.905847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.922958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:27.923599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.923876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.924145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.927206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:27.928625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:27.936638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:27.943868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:27.944835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:27.945698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 432/877] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:30:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:27.955507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.962617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.028393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 446/878] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:28.081637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b734-9df4-4e16-8e0e-61877c0fc869 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.124139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.125268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.126363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.126907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.127810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:28.128333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.129965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedb8d5-3152-4e7b-881b-dc86492bad5f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.139842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 433/879] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:28.146054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.227364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74beced-9919-4812-a716-461149b8c159 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.227364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74beced-9919-4812-a716-461149b8c159 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.227364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74beced-9919-4812-a716-461149b8c159 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.232736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 434/880] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:28.233872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.234155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.234442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.256146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:28.257931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:28.262766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.267442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60b29-19e5-422c-ab60-c1e27caa6de7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.312888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.315068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.317360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.317862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.329940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.331445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.341776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 435/881] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:28 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:28.342856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:28.346844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:28.347779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:28.348666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:28.348666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:28.348666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:28.352632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 447/882] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:28.353658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.374504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.474117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758989f-f795-4c01-9e18-c4ad69528ef5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.496871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cef6e-f9cd-4220-8033-6fbc1442b274 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.516891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.517831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.518525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.519259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.520099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.521647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.543353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.544938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.545588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.545882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.546192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.546918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:28.547251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.548985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.562987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 436/883] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:28 2022] POST /identity/v3/services => generated 269 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:28.566478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 448/884] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:28.573671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.633834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.708487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe944c-f2a5-4cc5-88a1-f49abb2957ec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.740592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.758048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.759025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.759511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.759989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.770109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.771605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.782185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11d62a-9120-4757-a042-4d520bb2d337 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:28.788533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:28.789516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:28.790550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:28.791180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 437/885] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:28.796396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.829669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.830523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.830814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.831086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.831679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.834353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:28.847484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:28.848743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:28.849841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 449/886] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:28.856183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:28.867261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:28.926399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972ae1c-0795-4dfe-ab5c-d3a859a868e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:28.972878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:28.974836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:28.976350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:28.977080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:28.978500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:28.979010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:28.980673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:28.994280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2af6f-0255-4f0a-8b80-bbf61e39e4e4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.005847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 438/887] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:29.012319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.035830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.036413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.036697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.036971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.037602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:29.037910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.039323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.041520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.050260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 450/888] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:29.051470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.139659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882787ac-e6dc-4d5f-ba3b-25d296e8345b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.158831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.163835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6ff5-fd96-4533-af7f-35c013ff1800 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.188905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.189741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.190123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.190542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.210979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.211599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.211886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.212161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.212794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:29.213100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.214819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a8e6-2c8e-4ab0-8a81-2d335c92cc40 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.227055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 451/889] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:29.282399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.284400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.369003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba90f9-3de1-443a-b364-1b9a7940e797 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.369324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba90f9-3de1-443a-b364-1b9a7940e797 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.369553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba90f9-3de1-443a-b364-1b9a7940e797 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.375326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 452/890] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:29.376996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.377275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.377548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.400116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:29.401877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:29.406058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 439/891] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:29.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.428304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.491584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 453/892] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:29.492281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.539738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7999-ae62-4c8a-b5bc-73c8b250f951 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.575965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.576935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.577454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.577949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.589348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.591114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:29.612880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:29.615460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:29.616424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:29.616424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:29.616424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:29.618816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 440/893] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:29.631622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.742324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3f983-306e-45e0-9b19-5fff0fbc72f3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.790400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.792714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.792714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.792714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.792714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:29.792714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.794218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.794685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.811795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 454/894] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:29.813601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:29.869616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.875174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:29.912030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a10ea8-a865-46f9-a194-bc86b6f2826c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.925910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411199d-e6fc-460a-a6bf-ecc0deaec843 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:29.965469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.966239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.967334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.967334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.976992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.980177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:29.981058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:29.981951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:29.982494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:29.982975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:29.983767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:29.985838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:30.003029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:30.004068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:30.004920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:30.011969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:30.013182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:30.014087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:30.022581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 441/895] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:30.023785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.024670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 455/896] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:30.032574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.184620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77400f7-0206-42ca-822d-3845afc80686 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.208030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0fc84-93c0-487a-92a8-f24e4424fdf8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.223704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:30.226300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:30.235471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.236404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.236815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.237217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.239055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:30.239591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.241277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.260881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 442/897] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:30.275841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.276052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.277414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.278272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.279046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.280750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:30.283111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.285679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.320418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 456/898] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:30.335521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.441913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169418a-03a8-471e-8644-4f432415a556 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.515325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.516857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.519414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.519918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.520768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:30.521285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.523152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.540260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 443/899] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:30.541480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.581646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e7c0e-4b00-4197-a95d-d7bc8a3f4df6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.659347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.661121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.663120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.663670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:30.663849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.664824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] RBAC: Adding query filter params (role.id=c51f1a64d462425189371698a8d2ae76, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=d5846f447c204206bde8acaa4160664e) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:30.666394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.667889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.682397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 457/900] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/role_assignments?user.id=d5846f447c204206bde8acaa4160664e&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=c51f1a64d462425189371698a8d2ae76 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:30.727317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b536b-69e7-4ef5-a918-b2356af4c489 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.770488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.771047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.771328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.771602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.772334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] RBAC: Adding query filter params (interface=public, service_id=5afe0df028a345af96f4996b46ecc24b, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:30.772647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.774071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c64c13-6954-4d39-ace5-7788df51f3ff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.783844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 444/901] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/endpoints?service_id=5afe0df028a345af96f4996b46ecc24b&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:30.789789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.843112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:30.875804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faad2-f575-4330-a80d-73aad6b50e72 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.876348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faad2-f575-4330-a80d-73aad6b50e72 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.876348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faad2-f575-4330-a80d-73aad6b50e72 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.877716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 445/902] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:30.883694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.883980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.884251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.927784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48139-a1d6-45e7-966a-d52f2e3fbc51 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.931385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:30.931899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:30.967770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.968761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.969797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.970271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.971108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.972854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.986773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 458/903] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:30 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:30:31.018380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 446/904] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:31.351131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:31.469974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f8a9-3daf-4d26-aada-6086184008a9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:31.473480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:31.547167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:31.547167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:31.547167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:31.547167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:31.547565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:31.547565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-660138a7-13e0-4a2d-98ae-a1cc39afe702 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:31.557491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:31.558528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:31.559709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:31.563220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 459/905] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 15:30:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:31.569918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:31.617670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffb4-53bc-4439-95c0-489da2d44012 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:31.618211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffb4-53bc-4439-95c0-489da2d44012 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:31.618662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffb4-53bc-4439-95c0-489da2d44012 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:31.620437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 460/906] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:31.632029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:31.632527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:31.632966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:31.645425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:31.662498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:31.664319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:31.747438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e5539-771d-4546-8fec-3fd3c00813e9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:31.803609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 461/907] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:31.820879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:31.821885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:31.822386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:31.822941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:31.823786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:31.824293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:31.826903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:31.833348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 447/908] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:31.839380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:31.949570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:31.958605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b4d7f-95f0-41a2-88aa-d0d9e325cc12 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:31.994332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:31.995225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:31.995657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:31.996090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:31.996871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:31.997353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:31.999075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:32.005409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 462/909] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:32.011026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:32.121320 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc222c6-7093-42d5-9656-ea597ed4bcbd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:32.139079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:32.157599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:32.160523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:32.161138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:32.161412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:32.161788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:32.162389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] RBAC: Adding query filter params (interface=public, service_id=fe4efb3269d14796acd2c834c7ff3858, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:32.162735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:32.164099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:32.177631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 448/910] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:32 2022] GET /identity/v3/endpoints?service_id=fe4efb3269d14796acd2c834c7ff3858&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:32.178580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:32.299977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6fdb-5b8c-495b-baba-02c66365d9aa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:32.302366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae1c0f-cc29-4995-8b25-b66b870f8e2c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:32.345622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:32.345622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:32.345622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:32.346597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:32.346876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:32.348999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891ec96-49ab-48ef-afd1-e8e529adc165 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:32.363449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 449/911] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:32 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:30:32.368422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:32.369080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:32.374532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:32.374532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:32.374532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:32.374532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f28717-c022-407e-8a69-28e735f7f1af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:32.396895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 463/912] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:32 2022] POST /identity/v3/services => generated 241 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:33.106914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacf19-606c-4c43-af35-03e6682a5df5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.107428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacf19-606c-4c43-af35-03e6682a5df5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.107428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacf19-606c-4c43-af35-03e6682a5df5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.108780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 450/913] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:33.115573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a19a-c707-4e11-b5d0-9ee072db3ef4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.115869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a19a-c707-4e11-b5d0-9ee072db3ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.116150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a19a-c707-4e11-b5d0-9ee072db3ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.143620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-12d9a19a-c707-4e11-b5d0-9ee072db3ef4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:33.143620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12d9a19a-c707-4e11-b5d0-9ee072db3ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:33.160352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:33.234928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 464/914] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:33.407743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e873-e473-45c0-af24-f704b2777509 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.408066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e873-e473-45c0-af24-f704b2777509 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.408285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e873-e473-45c0-af24-f704b2777509 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.409614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 451/915] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:33.415288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.415744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.415837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.439316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:33.440512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:33.532110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 465/916] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:33.559922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e873-e473-45c0-af24-f704b2777509 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:33.624012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:33.691659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e873-e473-45c0-af24-f704b2777509 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:33.726754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:33.727581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.727955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.728372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.739966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:33.745986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:33.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:33.747778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 452/917] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:30:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:33.759548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:33.827094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:33.865638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:33.892445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154d0d6-2ba5-410b-8d41-4bf1600614fb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:33.930113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:33.930756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.935086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.935086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.935086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:33.935086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:33.935086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:33.948509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 466/918] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:33.962438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.010439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41905-51fd-4a3d-a322-36f67cfc9a8e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.010439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:34.080365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.082274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.082829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.083334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.084121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.085677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:34.096732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:34.097769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:30:34.098642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:34.108053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 453/919] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:34.109888 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.122319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0da3f-a69f-4590-9ab9-bd491aa1983b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.169414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.170048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.170329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.170633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.179341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:34.185543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.187347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:34.209399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:34.210369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:34.211355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:34.211355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:34.211355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:34.215277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 467/920] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:34.220718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.255517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815273f4-5538-41b9-ad59-b44d313a5f16 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.300378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.300992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.301256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.301525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.302154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:34.303292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.304688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.315492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 454/921] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:34.316425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.365071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:34.376042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bc6ea-c25d-4a22-ad46-7f242e39389a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.432979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.434708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.436276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.436763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.438548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:34.440933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.442672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.465221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5288a9-a32f-49ed-960f-9b310a8b7eec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.466174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 468/922] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:34.467417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.506004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.506747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.507766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.508069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.514445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:34.514445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.519901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.533090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 455/923] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 15:30:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:34.535926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.585084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:34.690439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d21a-f0ea-4310-a1c8-ce4fa3708207 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.724102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:34.733582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2dcb1-a41d-47a3-a521-b182a59e9bb3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.749847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.750567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.753452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.753628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.767433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.768928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:34.791409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:34.792452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:34.793397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:34.794470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 469/924] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:34.805496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:34.809663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:34.809663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.809663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.809988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.810519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:34.816157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5e0a-2ee6-4bbc-a364-0eb727cd6b0b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:34.864629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 456/925] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:34.879123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84735a-0e8d-42f4-9d0d-c80a82c21179 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.879429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84735a-0e8d-42f4-9d0d-c80a82c21179 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.879700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84735a-0e8d-42f4-9d0d-c80a82c21179 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.881633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 457/926] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:34.889648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6e47-dfb7-48b9-8bac-de0f214d1982 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:34.889838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6e47-dfb7-48b9-8bac-de0f214d1982 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:34.890119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6e47-dfb7-48b9-8bac-de0f214d1982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:34.920039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5b5f6e47-dfb7-48b9-8bac-de0f214d1982 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:34.924366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b5f6e47-dfb7-48b9-8bac-de0f214d1982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:34.979774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e981e08-fc43-4dd4-8e37-ac5289c994aa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:34.992028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.032831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.033448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.033729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.034002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.034927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:35.035358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.037206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 458/927] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:35.037351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.038373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df9c4-e065-461b-9322-9a8d8e828ece None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.038718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df9c4-e065-461b-9322-9a8d8e828ece None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.038949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df9c4-e065-461b-9322-9a8d8e828ece None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.044283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 459/928] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:35.053651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.053651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.053651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.059402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 470/929] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:35.065272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.080120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:35.081574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:35.179917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 460/930] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:35.190529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733dc8e-352b-4845-8169-8be49f44653d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.231609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.232161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.234757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.234757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.254020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.255591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:35.274583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:35.275483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:35.276358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:35.276358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:35.276358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:35.276358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 471/931] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:35.281975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.301761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.336242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.371633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.420181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c7b24-6038-432c-b8c0-595766fad0fa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.467263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.468248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.468908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.469405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.470362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:35.470900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.472482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.490071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 461/932] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:35.492485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.506195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2186c2-d5f9-489b-b42a-36af62fb8e55 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.546210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.547928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.548618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.549119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.550278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.551892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:35.568445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:35.569638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:30:35.570856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:35.572082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.573083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 472/933] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:35.574089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.612206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1129e26-9b8b-476c-a5d1-d9ff4de8ce1a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.649089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.649782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.650058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.650370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.650994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.652504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:35.665356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:35.666519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:35.667422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 462/934] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:35.668225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.683548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.704696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfacd-eaef-4600-8348-5d19f678b949 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.746994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.747676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.748016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.748288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.758699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.761721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:35.774611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:35.775713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:35.776794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:35.778984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 473/935] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:35.780028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.785817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b938c1-c35a-4ec3-bf8d-a5a8598c81ce None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.830312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.831297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.831771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.832245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.832844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.833658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:35.834514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.836290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.847696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 463/936] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:35.849072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.914831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee29b-8ec4-45b4-9a12-b197026baee0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:35.954166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:35.954813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.955091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.955364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.955989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:35.956290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:35.957757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:35.964503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 474/937] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:35.966068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.975623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.986816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b667022-9bf0-4c53-92a5-23ceb5c37c74 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.032228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.032675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.032952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.033223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.034070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:36.034374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.036098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.057419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 464/938] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:36.058556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.102668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041312b4-9910-478d-aa5a-53fcc8522650 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.124442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:36.149395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.150161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.150581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.150954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.151697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:36.152103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.153603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.167625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 475/939] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:36.167625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.181943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67e80b-9b98-4762-97ee-2b449ffa91fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.230214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.231277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.231549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.231819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.232456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:36.232757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.234806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.241280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 465/940] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:36.242279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.258977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:36.298858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495792fe-e0fc-4329-9986-c9f22f96a311 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.339274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.340284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.340714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.340990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/d5846f447c204206bde8acaa4160664e/roles/c51f1a64d462425189371698a8d2ae76` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.345616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=d5846f447c204206bde8acaa4160664e, role_id=c51f1a64d462425189371698a8d2ae76)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.347097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.370599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 476/941] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:36 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/d5846f447c204206bde8acaa4160664e/roles/c51f1a64d462425189371698a8d2ae76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:36.370599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.384321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a801d-a889-4b9d-95de-b9130d927a2d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.417734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:36.445788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.446708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.447140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.447584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.448922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.451690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.485913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 466/942] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:36.556648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dcc33d-8d88-4e36-a1ac-89e58aedebd0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.590103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:36.610785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.611769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.612325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.612794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.613725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] RBAC: Adding query filter params (interface=public, service_id=3beae547840d4fae94b29fce0edc8837, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:36.614267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.615949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e737f8-a8ee-48d2-bbb7-7e71f7e3da9a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.622471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 477/943] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:36 2022] GET /identity/v3/endpoints?service_id=3beae547840d4fae94b29fce0edc8837&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:36.629437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.775586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6228-bb51-4548-9069-5e38b2a64bef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.817611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.817936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.818221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.819267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.819267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.820476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2b696-a400-4cc2-a647-ffae94eaa05d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.830761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 467/944] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:36 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:30:37.298547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48a83-6f60-4fad-9d20-8b8f3ff48920 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:37.298832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48a83-6f60-4fad-9d20-8b8f3ff48920 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:37.299105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48a83-6f60-4fad-9d20-8b8f3ff48920 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:37.300433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 478/945] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:37.306100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:37.306597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:37.306676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:37.344679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:37.346094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:37.389139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:37.433198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 468/946] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 28 15:30:37.723429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f48a83-6f60-4fad-9d20-8b8f3ff48920 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:37.836176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f48a83-6f60-4fad-9d20-8b8f3ff48920 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:37.878534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:37.878534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:37.878534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:37.878534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:37.879130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:37.879831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:37.882825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING keystone.server.flask.common [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 15:30:37.902862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.sql.core [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:30:37.902862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 15:30:37.902862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [parameters: {'id': '505f5b94f51844d8a7cb6dba9cc792f1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 15:30:37.902862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 505f5b94f51844d8a7cb6dba9cc792f1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 505f5b94f51844d8a7cb6dba9cc792f1. Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:30:37.921396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 28 15:30:37.922872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application yield resource Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise self.value Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application session.commit() Sep 28 15:30:37.923833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:30:37.924723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:30:37.925614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:30:37.926452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [parameters: {'id': '505f5b94f51844d8a7cb6dba9cc792f1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:37.927348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:37.928134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 505f5b94f51844d8a7cb6dba9cc792f1. Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:37.929093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 479/947] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Sep 28 15:30:37 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:37.929783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:38.042519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791387a7-d495-492a-92c4-b2059230c1ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:38.082463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:38.082463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.082463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.082463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.090807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:38.092460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5264998c-6f25-449a-b3ff-41f02835e5bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:38.104841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:38.105792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:38.106755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 469/948] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:38.112479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:38.163136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:38.225416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dc9fe-041f-4b8c-98d2-7544c80ec437 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:38.264051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:38.264724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.265006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.266100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.266924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:38.267257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:38.269038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec7e4-bdad-4b56-9a87-0ce1c8b0b278 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:38.291969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 480/949] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:38.952112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afd0b6-06ff-4cf3-9aae-cad45e93f00c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.952662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afd0b6-06ff-4cf3-9aae-cad45e93f00c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.952662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afd0b6-06ff-4cf3-9aae-cad45e93f00c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.958440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 470/950] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:38.960638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d2df-4f7f-4e1e-a1ed-c2e5ceeb0ed6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.961140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d2df-4f7f-4e1e-a1ed-c2e5ceeb0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.961227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d2df-4f7f-4e1e-a1ed-c2e5ceeb0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.987465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-14d8d2df-4f7f-4e1e-a1ed-c2e5ceeb0ed6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:38.990245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-14d8d2df-4f7f-4e1e-a1ed-c2e5ceeb0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:39.075924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 481/951] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:39.103772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38789-4549-4f37-94d1-185d03013ad7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.104050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38789-4549-4f37-94d1-185d03013ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.104317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38789-4549-4f37-94d1-185d03013ad7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.109679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 471/952] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:39.112541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030ef31-8844-4957-88a0-a4f4231a2434 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.112821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030ef31-8844-4957-88a0-a4f4231a2434 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.118528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030ef31-8844-4957-88a0-a4f4231a2434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.134338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e030ef31-8844-4957-88a0-a4f4231a2434 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:39.135560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e030ef31-8844-4957-88a0-a4f4231a2434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:39.164407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.228996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 482/953] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:39.334513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c86c-627d-4db5-9c83-c10b2e1bbd09 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.334829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c86c-627d-4db5-9c83-c10b2e1bbd09 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.335061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c86c-627d-4db5-9c83-c10b2e1bbd09 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.337370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 472/954] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:39.341747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.342033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.342309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.362536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:39.364032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:39.382789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8c86c-627d-4db5-9c83-c10b2e1bbd09 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.444166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.472322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 483/955] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:39.552039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.582637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8c86c-627d-4db5-9c83-c10b2e1bbd09 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.639524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.644879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.644879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.645153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.666066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:39.672869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:39.673847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:39.675188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 473/956] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:39.681557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.738657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.749837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ede252-f56c-46c1-85b6-4259071d0031 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.813448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.814090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.814716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.814967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.815914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.817450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:39.827432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:39.829119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:39.833800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 484/957] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:39.834803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.850279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.862798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0060a18-2dbd-4d13-a6aa-339b19edac1f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.902731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.903687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.903887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.904169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.904836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:39.905144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.907192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.920946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 474/958] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:39.921839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.951656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9364c-7079-4d2c-883f-71bc6feb257f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.996902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.998322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.998929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.999401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.000220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.002050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:40.012667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:40.013758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:40.014657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:40.018284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 485/959] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:40.019334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.055335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.067099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f682-8df0-4a3a-af72-59ca9d351f35 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.111989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.112951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.113299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.114806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.115546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:40.115918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.117361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.125229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 475/960] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:40.127471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.149048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.152159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c679-e1ac-4d59-8619-8b7f7a268f38 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.194250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.195044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.195476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.195818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.210780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.212325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:40.228269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:40.229282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:40.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:40.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:40.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:40.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 486/961] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:40.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.258294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fed87-6103-4dd6-89fe-aad8c1d7f374 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.308565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.309433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.309711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.309984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.310782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:40.311002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.312965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.323481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 476/962] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:40.324467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.335947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.371140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f691dd-652d-4ad5-bbcb-824e5659808d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.424044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.424955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.425079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.425268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.425919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:40.426223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.427925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.432824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.433560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 487/963] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:40.434832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.467733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bf16-93c2-4d08-be29-56700b52ca1e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.528567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.529478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.530155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.530637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.532033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:40.532506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.535281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.553359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 477/964] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:40.559091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.601140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35cb7a-0985-4b88-97f9-d3ae5f4fec88 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.662757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.663637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.663956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.664368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.665014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:40.665324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.667780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.674953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 488/965] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:40.681483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.684864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.699585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f29184-19f8-4fa8-8b63-4a0df472c9a3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.764859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.764859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.764859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.765757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.777178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.777178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.786088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:40.800029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:40.801047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:30:40.802069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:40.805195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 478/966] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:40.815957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:40.866508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138833-5f3a-4e44-98d5-906248371420 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.914766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:40.915722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.916105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.916475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.918479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:40.919278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acd9d4-160f-4907-88e2-7ed874baf365 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:40.953876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3928ac-4663-4a9a-8ab4-3d03f4f25a3e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:40.954684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 489/967] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:30:40.956492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ec84-615f-436d-bfa8-e811c99b2aa9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.956844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ec84-615f-436d-bfa8-e811c99b2aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.957183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ec84-615f-436d-bfa8-e811c99b2aa9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:40.962478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 490/968] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:40.965536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:40.965827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:40.966108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.007572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:41.010471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:41.058399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:41.061084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.061945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.061945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.063346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:41.063346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:41.065670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:41.091133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:41.112216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 479/969] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 15:30:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:41.113107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 491/970] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:41.119453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.179671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:41.227667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea56bb3-0b2b-4c22-b36d-3d02e430e181 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:41.285016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:41.286015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.286532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.287012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.372982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:41.374356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:41.426370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.447572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:41.487489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 480/971] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:41.493735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.584486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2724c8-7085-4a9c-8924-d42f59d71d15 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:41.606760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa964e59-5515-4d1b-ba2e-3372c8eec54d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:41.646325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:41.650435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.650435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.650435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.662439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:41.662439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:41.677143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:41.678204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:41.679451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:41.679451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:41.679451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:41.679451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 481/972] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:41.686071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.696514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:41.697090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:41.697994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.698535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.698992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.700020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:41.702659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:41.733018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.sql.core [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:30:41.733018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 15:30:41.733018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [parameters: {'user_id': 'e1f58c38f4cb466087a5dd719368b755', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 15:30:41.733018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:30:41.741314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:30:41.742490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application yield resource Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise self.value Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application session.commit() Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:30:41.743630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:30:41.744735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:30:41.745886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:30:41.746868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e1f58c38f4cb466087a5dd719368b755', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:41.747893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 28 15:30:41.748881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 15:30:41.750635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:41.753780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 492/973] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 15:30:41 2022] POST /identity/v3/users => generated 160 bytes in 328 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:41.759529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.821246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1e774-6cf3-4012-a3c5-8103cc205d29 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:41.853142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:41.870886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:41.871772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.872194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.872616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.873944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:41.874389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:41.876447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:41.894865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 482/974] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:41.901019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:41.972519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a1c1-5269-4458-94a2-bd8d9a8d102c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:41.988507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.035768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba988001-af68-431e-a007-a7f84b41cd8b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.038382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.039807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.039807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.039807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.051148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.052818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:42.073458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:42.074565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:30:42.075495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:42.081424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 493/975] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:42.082313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.083020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.083815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.084222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.084616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.093799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.096167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:42.111346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:42.112801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:42.113729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:42.116083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 483/976] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:42.131586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.274941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db2fe5-8ea1-4fca-b6ba-611375da0e1e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.282510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c40b8-9f68-4be4-b548-e61ae0cbe52d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.299159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.315595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.324066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.324680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.324959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.325231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.326603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:42.326910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.328880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.346381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 484/977] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:42.353773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.358504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.359532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.360137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.360718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.361631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:42.362177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.363879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13837745-b046-400b-b017-60bb4c27ecda None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.430530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 494/978] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:42.519189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ff4c-bfd2-42f4-8796-90dfb40ea0a1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.572595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.572595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.572595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.572595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.573024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] RBAC: Adding query filter params (role.id=c51f1a64d462425189371698a8d2ae76, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=d5846f447c204206bde8acaa4160664e) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:42.573024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.574114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14989a-a505-4dab-8713-264cb038f350 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.583741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 485/979] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/role_assignments?user.id=d5846f447c204206bde8acaa4160664e&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=c51f1a64d462425189371698a8d2ae76 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:42.894171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd5ad-6cd9-4930-81c7-58ee2717e74e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.894171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd5ad-6cd9-4930-81c7-58ee2717e74e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.894171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd5ad-6cd9-4930-81c7-58ee2717e74e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.895079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 495/980] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:42.903409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d264-4e47-40f7-aaf9-7f26d5c73af7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.903702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d264-4e47-40f7-aaf9-7f26d5c73af7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.903964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d264-4e47-40f7-aaf9-7f26d5c73af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.927362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4ad6d264-4e47-40f7-aaf9-7f26d5c73af7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:42.928537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4ad6d264-4e47-40f7-aaf9-7f26d5c73af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:42.987819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:43.016833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 486/981] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:43.297277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd5ad-6cd9-4930-81c7-58ee2717e74e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:43.419495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd5ad-6cd9-4930-81c7-58ee2717e74e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:43.460811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:43.461771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:43.462263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:43.462796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:43.463751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:43.465424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:43.468887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d67c5-7def-4a02-819c-1bae6552c643 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:43.469350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d67c5-7def-4a02-819c-1bae6552c643 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:43.469453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d67c5-7def-4a02-819c-1bae6552c643 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:43.474831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 487/982] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:43.476071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:43.476352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:43.476634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:43.497720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:43.498982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:43.500372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 496/983] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:43 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:43.656457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 488/984] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:43.925757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:44.004006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:44.102554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a36ae-10ba-47da-bcea-593ed571ac9d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:44.138407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:44.139072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:44.139355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:44.139625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:44.140767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:44.142127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:44.151326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:44.152436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:44.153283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 497/985] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:44.160252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:44.261300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1faf8-a607-4795-a9e8-aec9959ab84c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:44.274006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:44.298667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:44.299278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:44.299560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:44.299837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:44.300466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:44.300777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:44.302146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:44.307191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 489/986] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:44.313505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:44.406139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:44.416278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7b323-e7de-4d4a-be3e-e649ae588bd4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:44.453727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:44.454333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:44.454653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:44.454928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:44.455562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:44.455866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:44.457227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:44.462317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 498/987] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:44.468060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:44.572166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bced-2959-455c-9b80-619890a22545 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:44.606487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:44.607095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:44.607376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:44.607647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:44.608356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] RBAC: Adding query filter params (interface=admin, service_id=3beae547840d4fae94b29fce0edc8837, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:44.609204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:44.610593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e801c-840b-4716-8a8e-ea5dd53969b6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:44.615789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 490/988] 10.222.0.60 () {56 vars in 1249 bytes} [Wed Sep 28 15:30:44 2022] GET /identity/v3/endpoints?service_id=3beae547840d4fae94b29fce0edc8837&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:44.621270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:44.721243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c516-52b6-4d35-8f5c-e7fdb9a06066 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:44.755359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:44.755961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:44.756238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:44.756511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:44.757120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:44.758508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b40f1-9e07-40dc-a391-d46a0e2f8eeb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:44.770454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 499/989] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:44 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:30:45.058134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c9ad-3e9a-40a8-abc2-b3eb2c45ac82 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.058134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c9ad-3e9a-40a8-abc2-b3eb2c45ac82 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.058134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c9ad-3e9a-40a8-abc2-b3eb2c45ac82 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.059588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 491/990] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:45.064948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.065233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.065508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.086819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:45.088329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:45.104501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:45.170501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 500/991] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:45.452563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7c9ad-3e9a-40a8-abc2-b3eb2c45ac82 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.488874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:45.563829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7c9ad-3e9a-40a8-abc2-b3eb2c45ac82 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:45.597512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:45.598163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.598402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.598702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.599288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:45.600641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:45.609791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:45.611004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:45.611956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:45.619247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 492/992] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:45.619511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.720355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe199ceb-7b80-4497-8c20-23aae0673ed1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:45.754246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:45.754888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.755171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.755441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.756066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] RBAC: Adding query filter params (name=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:45.756371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:45.757731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefb4b9-80b4-4098-b737-a189d194a16c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:45.772527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 501/993] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:45.773911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.883971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:45.888766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f60036-f685-4622-a30d-36d7781cb6c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:45.926034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:45.926669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.926950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.927273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.927910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:45.928214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:45.929570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b37fc-da88-4352-819c-40a9f507d605 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:45.941389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 493/994] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:45.941613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c356f7-c2e0-4df1-9f0e-02ff50ec7c21 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.941613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c356f7-c2e0-4df1-9f0e-02ff50ec7c21 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.941613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c356f7-c2e0-4df1-9f0e-02ff50ec7c21 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.941613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 502/995] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:45.943726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.944012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.944284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.964069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:45.965298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:46.043827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 494/996] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:46.309700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c356f7-c2e0-4df1-9f0e-02ff50ec7c21 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.325131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.413979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c356f7-c2e0-4df1-9f0e-02ff50ec7c21 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.448707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.449333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.449613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.449885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.450489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.451862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:46.460825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:46.461855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:46.462744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:46.466488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 503/997] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:46.468791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.565429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.574096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6b580-460c-4452-b1d2-012147c75718 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.609693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.610374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.610628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.610887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.611461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:46.611764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.613130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.618290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 495/998] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:46.623970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.704850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.728147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3445bf-e6c4-4911-bc2f-cdcce2e6d643 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.763330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.763935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.764215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.764483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.765631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:46.765935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.767322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.772281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 504/999] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:46.778105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.885906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e778b7a-ac94-4d46-ac47-1b77e2d8b6c9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.922150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.922786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.923068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.923342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.924041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] RBAC: Adding query filter params (interface=public, service_id=ae5528c5c6be4013877a9fc2b9603773, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:46.924350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.925704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdef63-0ba0-459a-853b-d94f68364264 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.931059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 496/1000] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/endpoints?service_id=ae5528c5c6be4013877a9fc2b9603773&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:46.937474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.986763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549267d-3007-4050-bb3b-a66401cc2d62 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.987097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549267d-3007-4050-bb3b-a66401cc2d62 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.987323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549267d-3007-4050-bb3b-a66401cc2d62 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.988902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 497/1001] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:46.993842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.994126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.994445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.013064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:47.014254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:47.029529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:47.042895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0105-8a59-4dcf-834b-826a332c3546 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:47.077954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:47.078592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:47.078875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:47.079209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.079843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:47.081175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:47.087723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 505/1002] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:30:47.089993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 498/1003] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:47.363880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:47.373315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:47.465325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440fdd3-4b01-4bac-8566-ac535499005f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:47.500824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:47.501464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:47.501745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:47.502018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.502646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:47.504037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:47.514819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:47.516425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:30:47.517487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:47.518906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 506/1004] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 15:30:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:47.522406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:47.627847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dc3c7-7833-4244-b0f1-3ca67c8628d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:47.664593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:47.665212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:47.665490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:47.665742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.666395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:47.666725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:47.668090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:47.673900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 499/1005] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:47.679562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:47.751674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:47.784129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7a61-6131-4b49-a1b9-b55d0459a7df None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:47.819357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:47.819969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:47.820245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:47.820511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.821140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:47.821444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:47.822846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1eca8d-50d0-48fa-a48c-f5ff72c27a91 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:47.828945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 507/1006] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 15:30:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:47.834697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:47.852799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:47.942491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dcf6a-36e6-4598-8208-86df8e3f9bd2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:47.979375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:47.979993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:47.980274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:47.980547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:47.981238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:47.982645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c58ac-dea6-4c5f-9f50-b9634c31566f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:48.002228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 500/1007] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:48.128740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2464a0-53b9-4b31-b78d-9467fae25ba7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:48.128933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2464a0-53b9-4b31-b78d-9467fae25ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:48.129196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2464a0-53b9-4b31-b78d-9467fae25ba7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:48.130817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 508/1008] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:48.136510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb046530-5135-4b3c-9ae5-3f0db4a7d73b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:48.136810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb046530-5135-4b3c-9ae5-3f0db4a7d73b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:48.137051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb046530-5135-4b3c-9ae5-3f0db4a7d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:48.158366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fb046530-5135-4b3c-9ae5-3f0db4a7d73b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:48.159624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb046530-5135-4b3c-9ae5-3f0db4a7d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:48.234176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:48.242461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 501/1009] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:48.507255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2464a0-53b9-4b31-b78d-9467fae25ba7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:48.617805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2464a0-53b9-4b31-b78d-9467fae25ba7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:48.653571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:48.654625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:48.655118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:48.655612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:48.656464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:48.658052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e57d6-83fb-4ea5-80e3-4fc58a2247fe None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:48.661947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:48.675463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 509/1010] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:48 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:49.287437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963134-449b-4186-a4bb-307ce5403b06 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:49.287921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963134-449b-4186-a4bb-307ce5403b06 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:49.288034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963134-449b-4186-a4bb-307ce5403b06 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:49.289475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 502/1011] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:49.294598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:49.294996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:49.295173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:49.314171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:49.315400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:49.388831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 510/1012] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:49.656885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50963134-449b-4186-a4bb-307ce5403b06 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:49.758909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50963134-449b-4186-a4bb-307ce5403b06 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:49.793345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:49.793977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:49.794271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:49.794558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:49.795144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:49.796537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:49.805384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:49.806392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:30:49.807269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:49.808943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 503/1013] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:49.812698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:49.916340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2c77b-57cb-40a7-b057-d1b78346542a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:49.938434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:49.956703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:49.957313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:49.957593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:49.957865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:49.958520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:49.958827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:49.960213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:49.971753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 511/1014] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:49.971948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:50.035234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:50.078200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0c23a-5e1d-4da6-a00c-0f6e276e9edf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:50.130138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:50.130937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.131216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.131483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.132177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:50.132480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:50.133832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b64c8-a20b-40ef-863a-528c48d69764 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:50.139331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 504/1015] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 15:30:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:50.144865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:50.195264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aade7b-08c4-4076-9e32-574c542fff1c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.195445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aade7b-08c4-4076-9e32-574c542fff1c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.195724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aade7b-08c4-4076-9e32-574c542fff1c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.201788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 505/1016] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:50.202757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.203138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.203311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.223352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:50.224541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:50.280743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41391335-e86b-4027-ad90-5ba2a5fe619c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:50.315088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 506/1017] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:50.330068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:50.330708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.331046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.331319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.332014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:50.333376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:50.364453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 512/1018] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:50.588631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:50.647136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:50.693780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00804684-f49f-4965-9a0c-76705fa39566 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:50.741033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:50.741663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.741938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.742204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.743334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:50.744763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cbce8-248c-406b-83c4-31725c14751e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:50.746510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 507/1019] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:50.752223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:50.800056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:50.861396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1f06-c42c-4af8-86d2-0a5d2cc24bb8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:50.878362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541514-6964-4cc8-a0fc-3bb26a8433e9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.878767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541514-6964-4cc8-a0fc-3bb26a8433e9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.879106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541514-6964-4cc8-a0fc-3bb26a8433e9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.880508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 508/1020] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:50.885992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.886294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.886593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.899718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:50.900323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:50.900603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:50.900874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:50.901543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:50.903004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:50.906461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:50.907804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:50.931800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 513/1021] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:50 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:30:50.990639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 509/1022] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:51.257242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:51.350572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:51.361661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4a553-5caf-4dec-9aca-f6cdb96248ee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:51.397489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:51.398138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:51.398440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:51.398741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:51.399334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:51.400711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:51.410405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:51.411457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:51.412361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 514/1023] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 15:30:51 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:51.419290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:51.475587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:51.532084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b100-381a-4f23-99d7-87fb43f05453 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:51.567264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:51.567875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:51.568157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:51.568431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:51.569064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:51.569370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:51.570754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:51.580181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 510/1024] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:51.581358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:51.684685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297be66b-bd29-4d82-9141-836aef086033 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:51.718865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:51.719476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:51.719757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:51.720033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:51.720661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:51.720974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:51.722341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:51.732126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 515/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:51.733188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:51.834364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dffb01-914c-46e4-adc7-f168c9d72af1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:51.852574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:51.873822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:51.874453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:51.874738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:51.875019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:51.875918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] RBAC: Adding query filter params (interface=public, service_id=8b940fa5e6f944b68ba617cc2f369b73, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:51.876247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:51.877770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eec60-d724-420d-808e-56690a4b6932 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:51.888623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 511/1026] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:30:51 2022] GET /identity/v3/endpoints?service_id=8b940fa5e6f944b68ba617cc2f369b73&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:51.890720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:51.914382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:51.994784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786caa7-e503-4acc-862d-e46f5a65a6b5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:52.027665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:52.028258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:52.028536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:52.028807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:52.029406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:52.031383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e13571-a1a2-4ead-af6e-8fa8046e1055 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:52.038067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 516/1027] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:30:51 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:30:52.677901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff254-f740-4b67-8e23-ff45f2a3af15 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:52.678440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff254-f740-4b67-8e23-ff45f2a3af15 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:52.678547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff254-f740-4b67-8e23-ff45f2a3af15 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:52.679918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 512/1028] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:52.685105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62e24-ccce-4c63-a659-a82eb1df5963 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:52.685480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62e24-ccce-4c63-a659-a82eb1df5963 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:52.685661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62e24-ccce-4c63-a659-a82eb1df5963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:52.705245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-10f62e24-ccce-4c63-a659-a82eb1df5963 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:52.706446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-10f62e24-ccce-4c63-a659-a82eb1df5963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:52.773706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 517/1029] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:53.051448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ff254-f740-4b67-8e23-ff45f2a3af15 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.136608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a23b-1ee7-4931-ab29-c016a069991a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.136888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a23b-1ee7-4931-ab29-c016a069991a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.137166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a23b-1ee7-4931-ab29-c016a069991a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.138532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 518/1030] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:53.144077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.144363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.144640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.153112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ff254-f740-4b67-8e23-ff45f2a3af15 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.165018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:53.166216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:53.176494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:53.191748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.192374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.192654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.192929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.204212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:53.209045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:53.210019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:53.210930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 513/1031] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:53.215899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.241842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 519/1032] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:53.267020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:53.337816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156dad-d66b-4c45-8ff2-c83eb4d53d01 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.372278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.372891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.373169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.373443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.374139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:53.374463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.375829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.388795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 514/1033] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:53.390308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.490889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb77a61-638d-4c1f-ba2f-c753bdcf5fbd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.508519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.525365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.525990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.526272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.526584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.539686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.539686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.543267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:53.556735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:53.557738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:53.558954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:53.558954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:53.558954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:53.558954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 520/1034] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:53.563730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.626488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441b840-5bb1-4107-83f2-18766cb02def None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.627912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:53.661417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.662045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.662328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.662630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.663732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.665149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.667049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 515/1035] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:53.672285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.677746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c265c-798f-46f3-86db-13406d7cab27 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.714019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.715333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.715839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.716249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.717043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:53.717584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.719368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.733979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 521/1036] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:53.739578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.786198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce1907-2e42-4f21-9d29-dec558910a96 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.797709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:53.823202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.823812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.824097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.824373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.825018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.826469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.845165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeec623-3069-4c4e-a63b-271ea07d3715 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:53.850217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 516/1037] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:30:53 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:53.890988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:53.892092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:53.892593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:53.893066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:53.902249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:53.904130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:53.918834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:53.920095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:53.921338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:53.923361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 522/1038] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:53.929843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:53.964756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.047403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565e5a-c834-43d0-a1bd-d13a211a3085 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.084754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.085367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.085640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.085910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.086996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:54.086996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.088423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19784a80-dcec-4b67-b9e2-cf492d78faa8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.110126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 517/1039] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:30:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:54.116108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.224815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb359b1-2f56-4cab-bed8-f3c8ce644d9a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.241123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.269018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.269632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.269912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.270174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.284734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da890c-7b14-43ca-9cec-d50d7b7bd124 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.285021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da890c-7b14-43ca-9cec-d50d7b7bd124 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.285292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da890c-7b14-43ca-9cec-d50d7b7bd124 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.286809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 518/1040] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:54.291732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.292014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.292290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.311933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:54.313156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:54.343276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.344740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.388598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 519/1041] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:54.454831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 523/1042] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:54.457667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.473909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.562558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9c0e-90c6-4099-8262-6b17c8094895 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.597434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.598176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.598702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.599184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.611191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.612819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:54.627416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:54.628451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:54.634224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:54.634224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:54.634224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:54.634224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 520/1043] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:54.636118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.655547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.659161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.727832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.759090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aff775-a4ba-428b-8ae3-a3e87134c3c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.766116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390143ac-ab40-4ad1-8fba-fe1381fc9ae5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.795935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.796541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.796816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.797084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.797719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:54.798020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.799448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.801717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.802332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.802751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.802928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.803506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.804870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.812708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 524/1044] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:54.818586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:54.818781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:54.819744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:30:54.820749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:54.823548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 521/1045] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:54.826244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.900046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.925994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb96-6894-4cbc-afb5-ff0258f6de1c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.942883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccc538-c819-40b8-8e1b-b95a9bd9e650 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:54.964677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.965302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.965576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.965854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.972639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.974074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.975678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:54.982875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:54.984200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:54.984656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:54.985090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:54.985866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:54.985993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:54.987142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:54.988385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 525/1046] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:54.989397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:54.990940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:54.992766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:54.998012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 522/1047] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:55.003882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:55.101936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996b26-eb2a-4278-b750-175bda3bbf27 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:55.109596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fcb-82ef-4d65-9a0a-29071915afdf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:55.136983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:55.137592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:55.137868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:55.138134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:55.138844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:55.139148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:55.141310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:55.144126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:55.148662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:55.149282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:55.149572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:55.149837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:55.150509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:55.151036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:55.152387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:55.154783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 526/1048] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:55.157910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 523/1049] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:55.160827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:55.163656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:55.224800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:55.270452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba5108-32a3-4554-a035-05d0116ae13e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:55.280184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e8dbc-5a77-4ba0-a454-c1d0204ba42d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:55.306387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:55.307082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:55.307296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:55.307540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:55.308773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] RBAC: Adding query filter params (role.id=c51f1a64d462425189371698a8d2ae76, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:55.308892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:55.310211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ab073-1299-4519-b2eb-6d2d381adad8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:55.316338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 527/1050] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:55 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=c51f1a64d462425189371698a8d2ae76 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:55.332231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:55.332875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:55.333151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:55.333420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:55.334112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:55.335558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196926ab-e7f9-4fbb-a050-3767e5ec26ee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:55.362556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 524/1051] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:30:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:30:56.119981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6c75-105f-4bda-870e-bea41cbb6cfa None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:56.120619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6c75-105f-4bda-870e-bea41cbb6cfa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:56.120619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6c75-105f-4bda-870e-bea41cbb6cfa None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:56.122018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 528/1052] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:56.127414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:56.127832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:56.128004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:56.146592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:56.147782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:56.226519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 525/1053] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:56.497395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6c75-105f-4bda-870e-bea41cbb6cfa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:56.560603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:56.603916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6c75-105f-4bda-870e-bea41cbb6cfa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:56.641150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:56.641786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:56.642068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:56.642341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:56.652728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:56.658087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:56.659141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:56.660073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 529/1054] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:56.668130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:56.710022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:56.772364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86867b4c-b784-4f03-a966-9a343d79027b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:56.806297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:56.806943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:56.807226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:56.807500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:56.808158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:56.808715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:56.810072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:56.822925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 526/1055] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:56.824193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:56.930351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb019ab-6884-4a1f-b7c5-3958edbd0b78 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:56.951088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:56.966749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:56.968232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:56.968730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:56.969252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] PATH_INFO: `/v3/users/ef523122cfa542bbbabe5d539fc413df` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:56.984002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] RBAC: Authorizing `identity:get_user(user_id=ef523122cfa542bbbabe5d539fc413df)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:56.985837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:56.988128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 530/1056] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:56 2022] GET /identity/v3/users/ef523122cfa542bbbabe5d539fc413df => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:56.993509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:57.120379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f4ec-93b2-4965-b6ad-d13996b96c7e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:57.123349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:57.163612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:57.164529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.164685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.165059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.251439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:57.252878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115d44d-8b9e-4ec7-968d-7e4950530624 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:57.356094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 527/1057] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:30:57.365477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:57.478329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568170-5934-430a-9f3f-589d63897734 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.478834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568170-5934-430a-9f3f-589d63897734 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.479120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568170-5934-430a-9f3f-589d63897734 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.480759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 528/1058] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:57.486586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7234-8fb8-43c6-872d-321f7bbd8ddc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.487203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7234-8fb8-43c6-872d-321f7bbd8ddc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.487666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7234-8fb8-43c6-872d-321f7bbd8ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.489209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0989-844e-4c59-89fa-d87a82d9f4bb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:57.496121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:57.512995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-227c7234-8fb8-43c6-872d-321f7bbd8ddc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:57.514958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-227c7234-8fb8-43c6-872d-321f7bbd8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:57.527160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:57.527779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.528052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.528319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.529418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:57.530924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:57.532789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 531/1059] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:57 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:57.539055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:57.590458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 529/1060] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:57.592405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:57.601867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae067-45f4-45df-a585-81761a09eb86 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.602149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae067-45f4-45df-a585-81761a09eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.602653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae067-45f4-45df-a585-81761a09eb86 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.603976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 530/1061] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:30:57.609296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.609841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.610124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.634676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:57.635901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:57.649588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225ad9-47a6-4d3e-85ec-7659f5a89b23 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:57.687635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:57.688255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:57.688935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:57.689212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:57.690095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=2c743018ad5648999dd38abff1cfcbc8, user.id=ef523122cfa542bbbabe5d539fc413df) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:57.690437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:57.691899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:57.697897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 532/1062] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:57 2022] GET /identity/v3/role_assignments?user.id=ef523122cfa542bbbabe5d539fc413df&scope.project.id=2c743018ad5648999dd38abff1cfcbc8&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:57.711505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 531/1063] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:57.857578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:57.966108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3dbe-81cb-405e-aae0-c4033c2b5fee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:57.977597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:57.987066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.003656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.004298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.004573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.004842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.017255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:58.022053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:58.023169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:58.024115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 533/1064] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:30:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:58.029069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.074556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.122855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195ab71-082d-4322-a94e-35a4d508ee5b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.142150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3e9ab-6073-4dff-8130-0a7172993706 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.163107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.163765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.164316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.164316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.199258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.200182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.200582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.201007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.202124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:58.202637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.204952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.219469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 534/1065] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:58.225889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.270042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.271542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.276408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.334634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.354062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd0c4-3ef8-4a5d-9d4a-a0305346a760 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.380724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 532/1066] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:30:58.386926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.395207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.395782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.395782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.396005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.413148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.415676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:58.439347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:58.441015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:58.442684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:58.442684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:58.442684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:58.446004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 535/1067] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:58.446980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.521285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7130-084e-4650-a617-c5b55b5207e1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.541457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.559282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.559481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9862ac-3be0-484a-a49a-583422702bff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.559907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.560182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.560449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.571976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.573421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:58.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:58.591394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:58.593885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:58.593885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:58.593885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:58.593885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 533/1068] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:58.597871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.598793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.599407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.599690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.599965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.600617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:58.600932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.602334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.618335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 536/1069] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:58.619799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.624674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.716011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200fe59-8597-4e9d-962e-1a2c69d41fd6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.746790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4228-e90d-4e9c-9eb8-84613af6748c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.753501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.754121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.754407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.754709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.755362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:58.755670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.757303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.771173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 534/1070] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:58.794904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.796033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.796493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.796938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.806324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.808265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:30:58.821942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:58.823048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:30:58.824005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:58.827120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 537/1071] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:58.830841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:58.920862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:58.952367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05a0f-d20d-4303-ad66-160b48250473 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:58.990778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:58.991314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:58.991593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:58.991902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:58.992639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:58.992945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:58.994384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.014025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 535/1072] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:30:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:59.020475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:59.024302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:59.133762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721447e0-b0b2-4b16-87cf-f6111074f0b0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:59.167118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:59.167761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.168051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.168328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.178929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:59.180372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:59.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:59.202451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:30:59.203475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:59.203475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:30:59.203475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:59.203475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 538/1073] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:59.203714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:59.312674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64c18-a260-4804-b62a-113660c194c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:59.345192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:59.351201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:59.351820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.352099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.352374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.353025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:59.353330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:59.354755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.368324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 536/1074] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:30:59.374820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:59.439812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:59.486986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52508c-61cb-4bb8-927d-6ddabf9c23bd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:59.522378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:59.522948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.523298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.523572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.530286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:59.531760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:30:59.544172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:59.545768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:30:59.546742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:59.550563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 539/1075] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:30:59.553073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:59.651763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344eeb1-84b7-4c75-ae0b-040bdf252ee1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:59.685491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:59.686640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.686923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.687194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.687881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:59.688184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:59.689601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f0991-da0d-46f5-94e7-d966b9a716d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.701468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 537/1076] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:30:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:59.708349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:59.816675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948edf4f-3ce5-4c30-8c5b-12dd65a7b9e4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:59.836748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:59.856147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:59.856792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.857066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.857332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/c51f1a64d462425189371698a8d2ae76` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.859615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=6fcc9eec4e114f538c506178c744b49a, role_id=c51f1a64d462425189371698a8d2ae76)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:59.861049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff3f3-8376-4fe6-8443-59f35e619842 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:59.874996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 540/1077] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:59 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/6fcc9eec4e114f538c506178c744b49a/roles/c51f1a64d462425189371698a8d2ae76 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:30:59.926736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297d95-bee9-41f1-a682-e3d7d395f849 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.927018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297d95-bee9-41f1-a682-e3d7d395f849 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.927289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297d95-bee9-41f1-a682-e3d7d395f849 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.928720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 538/1078] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:59.935524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89855119-4764-4657-b831-0b2456f73a25 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:59.936327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89855119-4764-4657-b831-0b2456f73a25 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:59.936603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89855119-4764-4657-b831-0b2456f73a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:59.957884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-89855119-4764-4657-b831-0b2456f73a25 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:59.959209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-89855119-4764-4657-b831-0b2456f73a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:00.032560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 541/1079] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:00.302957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e297d95-bee9-41f1-a682-e3d7d395f849 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:00.358841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:00.417377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e297d95-bee9-41f1-a682-e3d7d395f849 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:00.454025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:00.454674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:00.454956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:00.455226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:00.465315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:00.470665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:00.471704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:00.472997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 539/1080] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:00.477744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89855119-4764-4657-b831-0b2456f73a25 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:00.574778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89855119-4764-4657-b831-0b2456f73a25 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:00.582181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:00.610861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:00.611465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:00.611742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:00.612011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:00.612673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:00.612977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:00.614346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:00.627352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 542/1081] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:00.628504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:00.733967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172762-788f-4fd5-a75b-2b908d7adde2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:00.747718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:00.768613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:00.769258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:00.769647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:00.769809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] PATH_INFO: `/v3/users/ef523122cfa542bbbabe5d539fc413df` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:00.771035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] RBAC: Authorizing `identity:get_user(user_id=ef523122cfa542bbbabe5d539fc413df)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:00.772495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:00.774169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 540/1082] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:00 2022] GET /identity/v3/users/ef523122cfa542bbbabe5d539fc413df => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:00.780907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:00.876475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4aa4b-7048-42c6-bb51-c8fca7fe3359 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:00.910909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:00.911524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:00.911798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:00.912067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:00.913161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:00.914600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014aa96-11ce-4abe-ad99-cb8acb07d354 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:00.916216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 543/1083] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:00 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:00.922162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:01.016405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195d0cd-8c2a-4828-84ba-0fee457a8b77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:01.048787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:01.049426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:01.049699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:01.049967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8/users/ef523122cfa542bbbabe5d539fc413df/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:01.051837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] RBAC: Authorizing `identity:create_grant(project_id=2c743018ad5648999dd38abff1cfcbc8, user_id=ef523122cfa542bbbabe5d539fc413df, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:01.053237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e46f1-b41d-4bbe-827d-cf2591deaddf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:01.066639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 541/1084] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:31:00 2022] PUT /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8/users/ef523122cfa542bbbabe5d539fc413df/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:31:02.070846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960b5d-bfbf-4eb6-9efb-09f033ac8c82 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:02.071425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960b5d-bfbf-4eb6-9efb-09f033ac8c82 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:02.071425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960b5d-bfbf-4eb6-9efb-09f033ac8c82 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:02.072723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 544/1085] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:02.078747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:02.079162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:02.079294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:02.097079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:02.098249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:02.131865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:02.165683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 542/1086] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:02.433640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960b5d-bfbf-4eb6-9efb-09f033ac8c82 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:02.469329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:02.532585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960b5d-bfbf-4eb6-9efb-09f033ac8c82 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:02.569953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:02.570618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:02.570903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:02.571176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:02.581656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:02.586457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:02.587576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:02.588703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 545/1087] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 15:31:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:02.594240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:02.692809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ecf22-2073-46ea-a940-12014bb1fb1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:02.726247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:02.726876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:02.727147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:02.727415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:02.728069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:02.728368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:02.729717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:02.742261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 543/1088] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:31:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:02.743447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:02.835596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:02.842872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8c5a4-27b6-48ce-8246-bdafb9dc663f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:02.877353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:02.878001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:02.878253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:02.878561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:02.888918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:02.890379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:02.905797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:02.906831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:02.907746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:02.907746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:02.907746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:02.907746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 546/1089] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:02.912664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:02.944121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.011646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64fa6-2bd6-4947-b3b4-b5982410d97c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.044760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.045396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.045680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.045951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.046623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:03.046928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.048280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37712980-183a-4208-a77d-7a959f11e732 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.059876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 544/1090] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:03.065753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.167457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f991450-fa33-43d8-ae33-694a3cb94df7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.200101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.200645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.200927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.201712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.208989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.210446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:31:03.221920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:03.222955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:31:03.224007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:03.224559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cd317-301c-42d1-9cd5-4d1ce8ff68bf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.224559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cd317-301c-42d1-9cd5-4d1ce8ff68bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.224559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cd317-301c-42d1-9cd5-4d1ce8ff68bf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.225086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 547/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:03.225168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 545/1092] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:31:03.228564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.229670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.229955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.230234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.240539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.251367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:03.252555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:03.320751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 546/1093] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:31:03.332138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f1ad7-bb8c-476c-8f35-518983003c80 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.364823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.365423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.365703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.365975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.366744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:03.367055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.368474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.385492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 548/1094] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:03.391943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.415273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.488808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f184eb-c2c9-463c-b516-21ac01a0cb1c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.521901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.522541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.522822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.523095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.590497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.591350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.591954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.615249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.655417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.690015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522144-ed10-411e-8a76-38cf60c33a01 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.690817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 547/1095] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:31:03.696619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.728271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.728885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.729158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.729428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.740113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:03.744851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:03.745965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:03.746931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 549/1096] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:03.752475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.791790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001c22-bfe8-4ed2-9e16-a5c977c85cf1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.814743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.826648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.827313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.827594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.827874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.837916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.839613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.851978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515085-358e-4fae-a7aa-e56047c8c170 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:03.854800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:03.855943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:03.856869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:03.856869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:03.856869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:03.856869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 548/1097] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:03.862612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.878842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:03.888171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:03.888818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:03.889096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:03.889373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:03.890041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:03.890348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:03.891826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:03.900971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 550/1098] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:03.906959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:03.969147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5c99f-dee4-423f-9cc5-cc3870a61753 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.002697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.003214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.003492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.003761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.004393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:04.004691 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.005490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694d33-e716-49cd-bea9-fbd08c7323c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.006044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.017489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 549/1099] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:04.023153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.037849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.038652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.038949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.039235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] PATH_INFO: `/v3/users/ef523122cfa542bbbabe5d539fc413df` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.040364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef523122cfa542bbbabe5d539fc413df)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.041789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.043398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 551/1100] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:03 2022] GET /identity/v3/users/ef523122cfa542bbbabe5d539fc413df => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:04.049493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.060381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:04.104513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:04.127854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd3eae-8d76-45a8-9662-3cd9dec786a4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.148431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6809e-daec-417c-b174-af3a77c272e1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.163021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.163650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.163921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.164190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.171268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.172686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.182439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.183060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.183598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.183871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:31:04.184187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:04.185086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:04.186029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 550/1101] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:04.191063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.252786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.254167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.289048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:04.296675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0389-0370-47d6-b05b-665b0c24d597 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.309198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:04.330949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.331556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.331836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.332106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.332791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:04.333214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.334647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.346407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 551/1102] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:04.352383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.358566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 552/1103] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:31:04.359537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.448063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db267-c831-49b3-ac2d-63875787a8f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.453104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390620-65cd-4422-9ee7-3cdbe052b63c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.480676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.481280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.481570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.481823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.483122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] RBAC: Adding query filter params (role.id=c51f1a64d462425189371698a8d2ae76, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=6fcc9eec4e114f538c506178c744b49a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:04.483427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.484843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.487504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.488124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.488406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.488684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.489793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.490461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 552/1104] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/role_assignments?user.id=6fcc9eec4e114f538c506178c744b49a&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=c51f1a64d462425189371698a8d2ae76 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:04.491246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b8dd9-1cfc-498a-a21e-d784365b1c9e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.493012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 553/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:04.499243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:04.520331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:04.604880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf86ee-ab8d-4421-906d-2fc2c72d6293 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:04.648405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:04.649031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:04.649308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:04.649582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:04.650352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=2c743018ad5648999dd38abff1cfcbc8, user.id=ef523122cfa542bbbabe5d539fc413df) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:04.650695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:04.652130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45183-b709-46aa-8acb-607771295906 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:04.659335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 553/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:04 2022] GET /identity/v3/role_assignments?user.id=ef523122cfa542bbbabe5d539fc413df&scope.project.id=2c743018ad5648999dd38abff1cfcbc8&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:06.853361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339c3b-4262-411c-b8b9-4d35f0b371d0 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:06.853894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339c3b-4262-411c-b8b9-4d35f0b371d0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:06.853894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339c3b-4262-411c-b8b9-4d35f0b371d0 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:06.855250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 554/1107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:06.861407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:06.861815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:06.861953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:06.880601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:06.881842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:06.946986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 554/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:07.222222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec339c3b-4262-411c-b8b9-4d35f0b371d0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:07.247797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:07.323207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec339c3b-4262-411c-b8b9-4d35f0b371d0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:07.355694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:07.356306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:07.356588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:07.356862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:07.358523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:07.359956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11407923-58f7-4638-ae4b-3fd0cb00aab2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:07.361517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 555/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:07.367442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:07.380891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:07.462285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc08f-8f8f-45b3-8364-c2ee69ed42eb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:07.496875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:07.497397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:07.497675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:07.497974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:07.498653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:07.500063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e5a35-51e2-4027-aecc-b3bcc38bc922 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:07.520651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 555/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:31:07 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:31:09.680057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022aa5-a973-4c4e-a3c3-da59255dfbad None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:09.680592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022aa5-a973-4c4e-a3c3-da59255dfbad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:09.680592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022aa5-a973-4c4e-a3c3-da59255dfbad None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:09.681941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 556/1111] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:09.688029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:09.688469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:09.688560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:09.707520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:09.708682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:09.766684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:09.773626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 556/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:10.042834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa022aa5-a973-4c4e-a3c3-da59255dfbad None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:10.132138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:10.141617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa022aa5-a973-4c4e-a3c3-da59255dfbad None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:10.174644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:10.175248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:10.175524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:10.175792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] PATH_INFO: `/v3/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:10.177151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] RBAC: Authorizing `identity:get_role(role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:10.178535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:10.179845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 557/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:10 2022] GET /identity/v3/roles/9f09201f0de74baf927082c77fc97d7b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:10.186066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:10.282794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234b2a-e8f5-489f-81e2-510516de1c6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:10.315444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:10.316000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:10.316278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:10.316549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] PATH_INFO: `/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:10.329109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] RBAC: Authorizing `identity:get_user(user_id=1636cb9f64cf4a82bdb5a9de3b1df3e2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:10.330579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:10.332161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 557/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:10 2022] GET /identity/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:10.338070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:10.440389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e40cf-1bac-4b8d-b666-9bf4cbe77c3f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:10.472626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:10.473226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:10.473504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:10.473777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:10.500405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:10.540821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:10.542262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:10.635314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 558/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:10.641675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:10.696748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:10.739941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c757428-9013-4c6e-93af-af5b28a7c780 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:10.777099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:10.777794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:10.778010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:10.778284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:10.779427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:10.780852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dafd0-9c03-48c0-b922-dd7c7782dc6c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:10.782502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 558/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:10 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:10.788797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:10.845761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:10.886569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8105-6104-4adb-9df9-00be02474b0f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:10.920668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:10.921265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:10.921545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:10.921817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:10.922569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=2c743018ad5648999dd38abff1cfcbc8, user.id=1636cb9f64cf4a82bdb5a9de3b1df3e2) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:10.922879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:10.924294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43912a7-402b-4db6-813e-61d1844d779c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:10.929532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 559/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:10 2022] GET /identity/v3/role_assignments?user.id=1636cb9f64cf4a82bdb5a9de3b1df3e2&scope.project.id=2c743018ad5648999dd38abff1cfcbc8&role.id=9f09201f0de74baf927082c77fc97d7b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:13.048693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db35ad-9f91-4d6f-bee8-71fbcbec0203 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.049223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db35ad-9f91-4d6f-bee8-71fbcbec0203 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.049342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db35ad-9f91-4d6f-bee8-71fbcbec0203 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.050654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 559/1118] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:13.055887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.056286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.056428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.074970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:13.076631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:13.141089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 560/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:13.412882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db35ad-9f91-4d6f-bee8-71fbcbec0203 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:13.510993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db35ad-9f91-4d6f-bee8-71fbcbec0203 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:13.534286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:13.545140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:13.545766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.546043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.546318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] PATH_INFO: `/v3/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.547733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] RBAC: Authorizing `identity:get_role(role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:13.549087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:13.550432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 560/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:13 2022] GET /identity/v3/roles/9f09201f0de74baf927082c77fc97d7b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:13.556319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:13.609252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:13.652458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49e45-9200-42dc-840e-e7b877e8318d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:13.686034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:13.686608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.686892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.687168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] PATH_INFO: `/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.688276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] RBAC: Authorizing `identity:get_user(user_id=1636cb9f64cf4a82bdb5a9de3b1df3e2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:13.689703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:13.691313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 561/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:13 2022] GET /identity/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:13.697132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:13.792488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63885a4-7a38-449f-8168-8efc85dafeb4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:13.829861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:13.830749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.831226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.831668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.833374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:13.835656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a668ee-8bdf-4489-8e48-e03eb1646355 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:13.838170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 561/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:13 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:13.844526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:13.942440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006c9f-ec25-497b-9a3a-68c3f0dc0d85 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:13.963818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:13.975817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:13.976456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:13.976735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:13.977009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8/users/1636cb9f64cf4a82bdb5a9de3b1df3e2/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:13.979094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c743018ad5648999dd38abff1cfcbc8, user_id=1636cb9f64cf4a82bdb5a9de3b1df3e2, role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:13.980525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68261c-64a5-40e5-ae37-5ffaf2d00d57 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:13.993434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 562/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:31:13 2022] PUT /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8/users/1636cb9f64cf4a82bdb5a9de3b1df3e2/roles/9f09201f0de74baf927082c77fc97d7b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:31:16.127876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9efb-5cb2-4f0b-8a98-7ef8ef279e44 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:16.128413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9efb-5cb2-4f0b-8a98-7ef8ef279e44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:16.128413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9efb-5cb2-4f0b-8a98-7ef8ef279e44 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:16.129739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 562/1124] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:16.135070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:16.135475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:16.135623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:16.153158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:16.154869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:16.218650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 563/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:16.487675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec9efb-5cb2-4f0b-8a98-7ef8ef279e44 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:16.544878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:16.584711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec9efb-5cb2-4f0b-8a98-7ef8ef279e44 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:16.617920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:16.618568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:16.618850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:16.619124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] PATH_INFO: `/v3/roles/9f09201f0de74baf927082c77fc97d7b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:16.620491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] RBAC: Authorizing `identity:get_role(role_id=9f09201f0de74baf927082c77fc97d7b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:16.621848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:16.623247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 563/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:16 2022] GET /identity/v3/roles/9f09201f0de74baf927082c77fc97d7b => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:16.629228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:16.723767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de26002-8c29-4f5f-b88d-3bb6ae29fc64 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:16.736285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:16.764005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:16.764609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:16.764974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:16.765247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] PATH_INFO: `/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:16.767743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] RBAC: Authorizing `identity:get_user(user_id=1636cb9f64cf4a82bdb5a9de3b1df3e2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:16.768538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:16.770901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 564/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:16 2022] GET /identity/v3/users/1636cb9f64cf4a82bdb5a9de3b1df3e2 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:16.776941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:16.880290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e47b8-65a5-4fe4-b23f-eede74c786af None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:16.906648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:16.914708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:16.915309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:16.915589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:16.915863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:16.981509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:16.983623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:17.076388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 564/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:31:17.082348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:17.190175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3adf5-de02-48e2-be70-435fd9a880f9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:17.222882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:17.223508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:17.223787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:17.224063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] PATH_INFO: `/v3/projects/2c743018ad5648999dd38abff1cfcbc8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:17.225176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] RBAC: Authorizing `identity:get_project(project_id=2c743018ad5648999dd38abff1cfcbc8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:17.226641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcbb9f-3e50-4d1c-91ff-322777457156 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:17.228279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 565/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:17 2022] GET /identity/v3/projects/2c743018ad5648999dd38abff1cfcbc8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:17.235092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:17.263729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:17.332526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b2fd-76eb-4be2-9707-ae9e75719f19 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:17.364648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:17.365243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:17.365522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:17.365794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:17.366546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] RBAC: Adding query filter params (role.id=9f09201f0de74baf927082c77fc97d7b, scope.project.id=2c743018ad5648999dd38abff1cfcbc8, user.id=1636cb9f64cf4a82bdb5a9de3b1df3e2) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:17.366857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:17.368271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07f092-c7bf-4532-bcf0-d9fdd9406c4b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:17.374356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 565/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:17 2022] GET /identity/v3/role_assignments?user.id=1636cb9f64cf4a82bdb5a9de3b1df3e2&scope.project.id=2c743018ad5648999dd38abff1cfcbc8&role.id=9f09201f0de74baf927082c77fc97d7b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:19.549260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce229-ce4d-470e-a54e-ba124e589830 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:19.549786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce229-ce4d-470e-a54e-ba124e589830 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:19.549786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce229-ce4d-470e-a54e-ba124e589830 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:19.551095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 566/1131] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:19.556404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:19.556836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:19.556932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:19.575556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:19.576767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:19.645677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 566/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:19.912135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce229-ce4d-470e-a54e-ba124e589830 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:19.963101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:20.013609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce229-ce4d-470e-a54e-ba124e589830 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:20.045417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:20.046031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:20.046312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:20.046613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:20.047714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:20.049139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aadbb-a4f5-4a5b-8fd3-e676559bd5a2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:20.050726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 567/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:20.056598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:20.080643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:20.154857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69848b50-f9c7-4faf-ac54-683fea77e879 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:20.187953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:20.188482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:20.188764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:20.189039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:20.189681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:20.191129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466febc6-f146-42b5-92aa-9162336b3129 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:20.217644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 567/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:31:20 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:22.372408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd846f-0906-4830-af6b-58bed27c3c62 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:22.373009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd846f-0906-4830-af6b-58bed27c3c62 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:22.373009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd846f-0906-4830-af6b-58bed27c3c62 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:22.374289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 568/1135] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:22.380376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:22.380805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:22.380927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:22.400098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:22.401280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:22.467259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 568/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:22.757842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd846f-0906-4830-af6b-58bed27c3c62 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:22.865216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd846f-0906-4830-af6b-58bed27c3c62 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:22.884091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:22.902478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:22.903629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:22.903910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:22.904182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:22.905308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:22.906783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcca15b-6783-4ca4-b0d3-e7d9e38d8984 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:22.908372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 569/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:22.914312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:22.917457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:23.020400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cd54-c241-480a-a1a5-7a9f0c1e892e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:23.053444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:23.054043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:23.054323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:23.054621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:23.055283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:23.056697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816cfac-3993-455d-94c0-b8744dab6fe3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:23.077654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 569/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:31:22 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:25.272295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12273b-44df-4ed7-bae0-e7f2f4564e69 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:25.272745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12273b-44df-4ed7-bae0-e7f2f4564e69 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:25.272857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12273b-44df-4ed7-bae0-e7f2f4564e69 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:25.274171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 570/1139] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:25.279860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:25.280279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:25.280415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:25.298798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:25.299974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:25.352903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:25.365351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 570/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:25.636543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12273b-44df-4ed7-bae0-e7f2f4564e69 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:25.734102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12273b-44df-4ed7-bae0-e7f2f4564e69 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:25.766338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:25.766965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:25.767242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:25.767509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:25.777123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:25.782435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:25.783533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:25.784592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 571/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:25.789492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:25.883440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4db15e-1be4-4e91-aa4e-71f0105ce6dc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:25.917050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:25.917650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:25.917924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:25.918194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:25.918895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:25.919194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:25.920544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:25.928181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 571/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:25.934861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:25.979416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:26.031782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748ff74-536b-453c-9d9d-64d0158fae68 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:26.064122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:26.064754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:26.065014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:26.065286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] PATH_INFO: `/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:26.078228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb5ebb427434826bbc2aef5bff6cd4c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:26.079695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:26.081280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 572/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:25 2022] GET /identity/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:26.087143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:26.139077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:26.185031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5888ebe-33e8-41a5-ad36-5e6add6d3ae8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:26.219155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:26.219754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:26.220042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:26.220316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:26.285037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:26.286506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:26.335679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:26.381334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 572/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:26.387901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:26.483089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba53fb3-ba90-4f94-83bc-bbc853c27263 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:26.484496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:26.516740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:26.517343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:26.517627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:26.517897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] PATH_INFO: `/v3/projects/3168c9c4a3144f769552d59c3a985623` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:26.519002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] RBAC: Authorizing `identity:get_project(project_id=3168c9c4a3144f769552d59c3a985623)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:26.520419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae8b9-ecfa-409e-b4da-1ee35123127c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:26.522022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 573/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:26 2022] GET /identity/v3/projects/3168c9c4a3144f769552d59c3a985623 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:26.528801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:26.636364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c58355-63ba-4dad-90d8-05d20420610c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:26.668419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:26.669010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:26.669284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:26.669553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:26.670265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=3168c9c4a3144f769552d59c3a985623, user.id=5fb5ebb427434826bbc2aef5bff6cd4c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:26.670598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:26.672000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bb17b-ecc2-4231-bf53-8646b2baa602 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:26.677084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 573/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:26 2022] GET /identity/v3/role_assignments?user.id=5fb5ebb427434826bbc2aef5bff6cd4c&scope.project.id=3168c9c4a3144f769552d59c3a985623&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:28.793735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00aef-0cbb-4448-acbf-6bac44b7d0ae None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:28.794284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00aef-0cbb-4448-acbf-6bac44b7d0ae None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:28.794381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00aef-0cbb-4448-acbf-6bac44b7d0ae None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:28.795699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 574/1147] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:28.800809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:28.801203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:28.801354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:28.819118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:28.820274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:28.834087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:28.890385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 574/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:29.157687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00aef-0cbb-4448-acbf-6bac44b7d0ae None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:29.263742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00aef-0cbb-4448-acbf-6bac44b7d0ae None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:29.313101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:29.313721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:29.314002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:29.314275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:29.324148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:29.329415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:29.330657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:29.331937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 575/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:29.336650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:29.430640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f1401-d30f-44b8-b407-0bce2bfc7fe8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:29.462637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:29.463234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:29.463506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:29.463772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:29.464426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:29.464726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:29.466719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:29.474393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 575/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:29.480276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:29.510255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:29.581141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67af3-4a8d-468d-8943-d026d76524c8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:29.615539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:29.616156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:29.616438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:29.616713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] PATH_INFO: `/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:29.617830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb5ebb427434826bbc2aef5bff6cd4c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:29.619286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:29.620867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 576/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:29 2022] GET /identity/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:29.627228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:29.638858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:29.723256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad0d52-bf8d-41ef-8a08-5321308b1ccb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:29.754996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:29.755606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:29.755884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:29.756152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] PATH_INFO: `/v3/projects/3168c9c4a3144f769552d59c3a985623` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:29.757250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] RBAC: Authorizing `identity:get_project(project_id=3168c9c4a3144f769552d59c3a985623)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:29.758683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec35ca-5eec-426c-8128-5a5b564ea49a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:29.760290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 576/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:29 2022] GET /identity/v3/projects/3168c9c4a3144f769552d59c3a985623 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:29.767026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:29.857468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:29.861050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08c66-031b-4ee6-9680-79f9801172e3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:29.894246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:29.894915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:29.895191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:29.895464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] PATH_INFO: `/v3/projects/3168c9c4a3144f769552d59c3a985623/users/5fb5ebb427434826bbc2aef5bff6cd4c/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:29.897320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] RBAC: Authorizing `identity:create_grant(project_id=3168c9c4a3144f769552d59c3a985623, user_id=5fb5ebb427434826bbc2aef5bff6cd4c, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:29.898768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f940fe9-0dcc-4907-8137-f06b31aa9a8e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:29.912203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 577/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:31:29 2022] PUT /identity/v3/projects/3168c9c4a3144f769552d59c3a985623/users/5fb5ebb427434826bbc2aef5bff6cd4c/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:31:32.046083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39388-338a-44c2-8b8d-41245d1e1aa1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.046683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39388-338a-44c2-8b8d-41245d1e1aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.046683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39388-338a-44c2-8b8d-41245d1e1aa1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:32.047992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 577/1154] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:32.053390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.053809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.053931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:32.071737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:32.072900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:32.136459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 578/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:32.409689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b39388-338a-44c2-8b8d-41245d1e1aa1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:32.486377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:32.509203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b39388-338a-44c2-8b8d-41245d1e1aa1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:32.542539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:32.543155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.543433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.543703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:32.553968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:32.562178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:32.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:32.564396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 578/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:32.569766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:32.654019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:32.666591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07468ad4-f8f4-48f6-9149-7148dc02112b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:32.699081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:32.699672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.699946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.700211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:32.701486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:32.701787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:32.703159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:32.710879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 579/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:32.716899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:32.812592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aa96d-4b1b-4a15-b7a9-1ec49966c27e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:32.845826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:32.846457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.846765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.847237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] PATH_INFO: `/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:32.848813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb5ebb427434826bbc2aef5bff6cd4c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:32.850228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:32.851831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 579/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:32 2022] GET /identity/v3/users/5fb5ebb427434826bbc2aef5bff6cd4c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:32.857765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:32.957158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bd32a-c1c8-4c25-b539-50a6065297ee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:32.988939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:32.989535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:32.989815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:32.990084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:33.008119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:33.054842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:33.056275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:33.150899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 580/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:33.157056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:33.160834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:33.253524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b152a-286e-49e9-a3c8-002d7f775201 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:33.286760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:33.287381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:33.287665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:33.287940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] PATH_INFO: `/v3/projects/3168c9c4a3144f769552d59c3a985623` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:33.289044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] RBAC: Authorizing `identity:get_project(project_id=3168c9c4a3144f769552d59c3a985623)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:33.290498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a849f-8fd0-4572-a045-b797e22f0c8d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:33.292111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 580/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:33 2022] GET /identity/v3/projects/3168c9c4a3144f769552d59c3a985623 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:33.298883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:33.346390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:33.394330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c866c2-1ed2-4647-9a86-6656e7f2db4b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:33.428046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:33.428638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:33.428915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:33.429183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:33.429905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=3168c9c4a3144f769552d59c3a985623, user.id=5fb5ebb427434826bbc2aef5bff6cd4c) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:33.430212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:33.431664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496c72-6631-40f8-927c-ef7be6e099d5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:33.437543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 581/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:33 2022] GET /identity/v3/role_assignments?user.id=5fb5ebb427434826bbc2aef5bff6cd4c&scope.project.id=3168c9c4a3144f769552d59c3a985623&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:35.575381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efacf3a-1e56-45a3-b3bb-c126beb78183 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:35.575899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efacf3a-1e56-45a3-b3bb-c126beb78183 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:35.575899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efacf3a-1e56-45a3-b3bb-c126beb78183 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:35.577187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 581/1162] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:35.582406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:35.582828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:35.582982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:35.601256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:35.602438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:35.666296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 582/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:35.941210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efacf3a-1e56-45a3-b3bb-c126beb78183 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:36.009105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:36.039593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efacf3a-1e56-45a3-b3bb-c126beb78183 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:36.071214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:36.071805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:36.072082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:36.072353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:36.137642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:36.139615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:36.204908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:36.234110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 582/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:36.240243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:36.286484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:36.339574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cc2a4-8ced-4041-84c1-5eb445049119 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:36.371419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:36.372037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:36.372316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:36.372586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:36.379590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:36.381023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b934fb4c-8c59-44d0-a2ca-c26ce7ce1950 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:36.392682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:36.393874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:36.395002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 583/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:31:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:36.399990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:36.496116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce947-67a9-489e-879e-48a888cccee2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:36.528140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:36.528735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:36.529510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:36.529786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:36.530436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:36.530753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:36.532105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0161c9-f193-4774-80cf-3e5b0e7e3d64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:36.538909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 583/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 15:31:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:38.672543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138383fa-35f1-40fb-a4b5-b4077a3abfa4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:38.673085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138383fa-35f1-40fb-a4b5-b4077a3abfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:38.673085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138383fa-35f1-40fb-a4b5-b4077a3abfa4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:38.674547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 584/1167] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:38.679766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532105-a15e-42ff-a0bb-9883c0e0489b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:38.680149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532105-a15e-42ff-a0bb-9883c0e0489b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:38.680331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532105-a15e-42ff-a0bb-9883c0e0489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:38.699490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f1532105-a15e-42ff-a0bb-9883c0e0489b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:38.700669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f1532105-a15e-42ff-a0bb-9883c0e0489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:38.728529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:38.765848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 584/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:31:39.035037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138383fa-35f1-40fb-a4b5-b4077a3abfa4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:39.131782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138383fa-35f1-40fb-a4b5-b4077a3abfa4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:39.134032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:39.164166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:39.164755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:39.165031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:39.165298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:39.165956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:39.167331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eca198-2877-4c54-b634-964ca2477c2a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:39.183401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 585/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Wed Sep 28 15:31:39 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:41.378290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff6596-cebc-46fb-ace8-c6df270f9c64 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:41.378876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff6596-cebc-46fb-ace8-c6df270f9c64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:41.378876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff6596-cebc-46fb-ace8-c6df270f9c64 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:41.380205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 585/1170] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:41.385627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:41.386028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:41.386168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:41.406210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:41.407429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:41.471396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 586/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:41.741037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff6596-cebc-46fb-ace8-c6df270f9c64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:41.838323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff6596-cebc-46fb-ace8-c6df270f9c64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:41.870224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:41.870875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:41.871155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:41.871426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] PATH_INFO: `/v3/domains/71dc6823c0ab457e925d2c7159d5fc4a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:41.882169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=71dc6823c0ab457e925d2c7159d5fc4a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:41.883627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5bc67-b281-4d82-90b0-5d518f482b48 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:41.885216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 586/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Sep 28 15:31:41 2022] GET /identity/v3/domains/71dc6823c0ab457e925d2c7159d5fc4a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:41.891817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:41.986035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:41.996198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e29b2-10dd-45aa-8461-35be2b256ae1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:42.038894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:42.039494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:42.039775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:42.040047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:42.040708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:42.042135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0272593-895c-4863-8969-6e62e8336d7d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:42.068519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 587/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:31:41 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:44.458269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56388c6-e65b-4934-9322-fc50259cdc2b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:44.458860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56388c6-e65b-4934-9322-fc50259cdc2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:44.458860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56388c6-e65b-4934-9322-fc50259cdc2b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:44.460153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 587/1174] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:44.465630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:44.466146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:44.466791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:44.485188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:44.486393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:44.550374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 588/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:44.817218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56388c6-e65b-4934-9322-fc50259cdc2b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:44.844870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:44.922712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56388c6-e65b-4934-9322-fc50259cdc2b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:44.956101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:44.956725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:44.957005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:44.957277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] PATH_INFO: `/v3/domains/71dc6823c0ab457e925d2c7159d5fc4a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:44.958389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=71dc6823c0ab457e925d2c7159d5fc4a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:44.959981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd18ec-c096-4fee-8da4-d0fd6212d28a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:44.961557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 588/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Sep 28 15:31:44 2022] GET /identity/v3/domains/71dc6823c0ab457e925d2c7159d5fc4a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:44.967368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:45.042734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:45.074872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77683e-eae4-4473-8f13-7355e63070b7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:45.124379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:45.124991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:45.125375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:45.125611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:45.126313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:45.128080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3eb6-0f0a-446d-955d-ebdc06d2a509 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:45.149594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 589/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:31:44 2022] POST /identity/v3/users => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:47.373299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317d89d-8d10-4425-962f-511f4aab74ef None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.373826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317d89d-8d10-4425-962f-511f4aab74ef None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.373917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317d89d-8d10-4425-962f-511f4aab74ef None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.375260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 589/1178] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:47.380650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.381088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.381203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.399531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:47.400704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:47.467900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 590/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:47.736678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317d89d-8d10-4425-962f-511f4aab74ef None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.829082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:47.836962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317d89d-8d10-4425-962f-511f4aab74ef None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:47.869130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:47.869751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.870029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.870302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.880528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:47.886214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:47.887224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:47.888622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 590/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:47.895268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.953728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:47.991958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b381-0b90-4a92-a2ef-202c37675de2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.024238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.024829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.025110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.025379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.026045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:48.026348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.027736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.035625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 591/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:48.041614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.138019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580292b-1a45-4d39-94ba-27126acd70c8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.170851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.171475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.171753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.172026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] PATH_INFO: `/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.186405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] RBAC: Authorizing `identity:get_user(user_id=d855bd0c15df49c5b2861a2a9f86f6f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.187871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.189460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 591/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:48.195299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.291755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68723-049c-4907-a9d3-76c98c23d977 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.315454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.326019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.326661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.326937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.327207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.391543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.392968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.487169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 592/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:48.493148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.507601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.590316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1e68-3034-4457-a8b5-dbdaf6211a8d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.625072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.625692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.625964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.626234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] PATH_INFO: `/v3/projects/2810848564dc428e92f323da885315d1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.627345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] RBAC: Authorizing `identity:get_project(project_id=2810848564dc428e92f323da885315d1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.628755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708b0f6-9bd4-45f1-84cd-a4d22724c3bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.630370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 592/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/projects/2810848564dc428e92f323da885315d1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:48.636565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.661131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.734032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b7af3-dbce-420b-9d8d-7880288b1383 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.766703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.767292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.767569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.767841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.768561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=2810848564dc428e92f323da885315d1, user.id=d855bd0c15df49c5b2861a2a9f86f6f0) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:48.768870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.770285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43f51-04c0-4e31-90c1-5c129ec78a20 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.775564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 593/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/role_assignments?user.id=d855bd0c15df49c5b2861a2a9f86f6f0&scope.project.id=2810848564dc428e92f323da885315d1&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:50.888225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edbba-aa7b-4642-9dfa-6a0d2c856a29 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:50.888814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edbba-aa7b-4642-9dfa-6a0d2c856a29 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:50.888814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edbba-aa7b-4642-9dfa-6a0d2c856a29 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:50.890089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 593/1186] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:50.895578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:50.896020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:50.896143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:50.917287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:50.918584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:50.986646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 594/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:31:51.251634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edbba-aa7b-4642-9dfa-6a0d2c856a29 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.331285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.350750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edbba-aa7b-4642-9dfa-6a0d2c856a29 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.383714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.384336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.384618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.384891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.395269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:51.399982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:51.401095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:51.402175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 594/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:51.407233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.424865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.504319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5da7-185e-412f-b264-ad7fd910dd68 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.537692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.538286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.538596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.538869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.539525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:51.539833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.541224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.549780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 595/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:51.555820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.651896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696fb0-d57f-48a8-8946-f472e5cc9255 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.685346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.685964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.686246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.686548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] PATH_INFO: `/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.687660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d855bd0c15df49c5b2861a2a9f86f6f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.689081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.690667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 595/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:51.696631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.775010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.794975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66eaea-68d3-4aaf-ba05-b1fc29591fdc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.828698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.829757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.830233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.830551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] PATH_INFO: `/v3/projects/2810848564dc428e92f323da885315d1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.831678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=2810848564dc428e92f323da885315d1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.833595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2335a-c59e-4ba4-a852-25348c55d6a6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.838446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 596/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/projects/2810848564dc428e92f323da885315d1 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:51.845253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.856201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.944169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51958a-48ab-4c7d-a70c-8c5621d331f4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.977286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.977932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.978212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.978517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] PATH_INFO: `/v3/projects/2810848564dc428e92f323da885315d1/users/d855bd0c15df49c5b2861a2a9f86f6f0/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.980402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] RBAC: Authorizing `identity:create_grant(project_id=2810848564dc428e92f323da885315d1, user_id=d855bd0c15df49c5b2861a2a9f86f6f0, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.981840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5584f-ffda-41b5-be98-ef2438479d3b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.995193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 596/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:31:51 2022] PUT /identity/v3/projects/2810848564dc428e92f323da885315d1/users/d855bd0c15df49c5b2861a2a9f86f6f0/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:31:54.118627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3798-a6c6-40e4-92af-8ebb442d9fac None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:54.119185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3798-a6c6-40e4-92af-8ebb442d9fac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:54.119185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3798-a6c6-40e4-92af-8ebb442d9fac None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:54.120505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 597/1193] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:54.126557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:54.127028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:54.127122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:54.144810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:54.145971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:54.204153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:54.210135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 597/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:54.479524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd3798-a6c6-40e4-92af-8ebb442d9fac None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:54.580387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd3798-a6c6-40e4-92af-8ebb442d9fac None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:54.613695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:54.614307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:54.614620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:54.614891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:54.624740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:54.629481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:54.630628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:54.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 598/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:54.636641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:54.732179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1211a-7eec-498e-b98a-54f4015e8d5e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:54.764393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:54.764991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:54.765273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:54.765549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:54.766206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:54.766543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:54.767906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:54.775629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 598/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:54.782223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:54.793174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:54.891043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1435a-2501-4495-9032-30fd8f79f48c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:54.927643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:54.928262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:54.928542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:54.928817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] PATH_INFO: `/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:54.929926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] RBAC: Authorizing `identity:get_user(user_id=d855bd0c15df49c5b2861a2a9f86f6f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:54.931385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:54.933225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 599/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 15:31:54 2022] GET /identity/v3/users/d855bd0c15df49c5b2861a2a9f86f6f0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:54.940696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:54.998543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:55.038608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1c94-46fe-457e-a70a-522bf7c2e1ab None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:55.070794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:55.071395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.071677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.071949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:55.136669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:55.138143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:55.197887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:55.238546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 599/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:55.244559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:55.318623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:55.340880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56701e9a-6947-4e5e-9eab-5ea53190a9fc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:55.373017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:55.373634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.373909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.374177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] PATH_INFO: `/v3/projects/2810848564dc428e92f323da885315d1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:55.375297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] RBAC: Authorizing `identity:get_project(project_id=2810848564dc428e92f323da885315d1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:55.376718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84453f4-9ceb-4bb2-9dd3-5d76fe23d595 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:55.378325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 600/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 15:31:55 2022] GET /identity/v3/projects/2810848564dc428e92f323da885315d1 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:55.384855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:55.482493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff85fde-dc73-4a5c-8bc3-97885636eaf8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:55.514854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:55.515387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.515666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.515944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:55.516665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=2810848564dc428e92f323da885315d1, user.id=d855bd0c15df49c5b2861a2a9f86f6f0) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:55.516970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:55.518384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd9f21-5b74-4e9a-8962-0ba09daecdf9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:55.523918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 600/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:55 2022] GET /identity/v3/role_assignments?user.id=d855bd0c15df49c5b2861a2a9f86f6f0&scope.project.id=2810848564dc428e92f323da885315d1&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:05.369266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=28 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:05.521451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:51.865497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb459-48e3-48b8-a35a-e1ec517502a6 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:51.866105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb459-48e3-48b8-a35a-e1ec517502a6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:51.866105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb459-48e3-48b8-a35a-e1ec517502a6 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:51.867809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 601/1201] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:51.873545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:51.874078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:51.874191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:51.897077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:51.898527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:51.976284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 601/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:52.249726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cb459-48e3-48b8-a35a-e1ec517502a6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:52.367320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cb459-48e3-48b8-a35a-e1ec517502a6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:52.415466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:52.416311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:52.416602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:52.416896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:52.429615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:52.431733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:52.451287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:52.452790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:52.453991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:52.453991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:52.453991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:52.453991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 602/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:52.463715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:52.588985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66088113-e444-44d6-8eaa-a49298c04d37 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:52.628459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:52.636764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:52.637553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:52.637837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:52.638112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:52.638801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:52.639111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:52.640486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f66b9-a1ca-4c6b-ae6b-4fc423e85efe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:52.654003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 602/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:52.659790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:52.726342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:52.791169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90e464-9d0c-44a8-8128-1bb27e0d16cf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:52.833966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:52.834891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:52.835332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:52.835756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:52.837186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:52.839085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f0d78-2583-4a4f-a92c-f53f59762afa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:52.864675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 603/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 15:32:52 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:32:55.189440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769405-06bf-42d7-bca1-9e40fd8fae20 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:55.190005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769405-06bf-42d7-bca1-9e40fd8fae20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:55.190005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769405-06bf-42d7-bca1-9e40fd8fae20 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:55.191476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 603/1206] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:55.197433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:55.197912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:55.198014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:55.218718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:55.220038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:55.306506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 604/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:55.582883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9769405-06bf-42d7-bca1-9e40fd8fae20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:55.713525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9769405-06bf-42d7-bca1-9e40fd8fae20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:55.753630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:55.755767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:55.756051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:55.756326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:55.776373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:32:55.783139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:55.784117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:55.785131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 604/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:55.793439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:55.844853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:55.920317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0e97-8ee8-4464-9322-a2c6bae04a52 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:55.972930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:55.973784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:55.974260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:55.974761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:55.975928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:55.976458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:55.979197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:55.992339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 605/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:55.999491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:56.059456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:56.115181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e882d-d7a6-4e51-a57f-8e34c4797e8d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:56.156155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:56.156887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:56.157175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:56.157431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:56.169215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:56.170807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:56.185774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:56.192212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:56.193225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:56.193225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:56.193225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:56.193225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 605/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:56.193471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:56.312283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:56.318587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe706-ea03-406c-9fe0-2e96994d7aab None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:56.358400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:56.359073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:56.359350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:56.359635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:56.360262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:56.360589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:56.362059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:56.374904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 606/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:56.380744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:56.490731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff2124-af80-4156-a1c1-955212e41310 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:56.492494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:56.530148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:56.530959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:56.531245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:56.531530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:56.540886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:56.542373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:32:56.554290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:56.561353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:56.562566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 606/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:56.563670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:56.679687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a06c16-46cd-4e32-b6dc-f567d76d6112 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:56.719795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:56.720543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:56.720856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:56.721259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:56.722001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:56.722311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:56.723948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:56.746666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 607/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:32:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:56.753290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:56.902554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f24a-c59e-409c-8912-f471014615a0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:56.950905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:56.951458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:56.951738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:56.952005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:56.982929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:57.022125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:57.023616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:57.158316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 607/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:32:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:32:57.164116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:57.189168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:57.306360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d124-ab7f-4ae2-89b8-b0892cb28b0b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:57.349337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:57.350061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:57.350281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:57.350633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:57.364549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:57.366041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:57.384857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:57.385947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:57.386896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:57.386896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:57.386896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:57.386896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 608/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:32:57.393434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:57.458317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:57.501433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9333b-7e14-4ac7-9deb-fbb52437536a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:57.542135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:57.542714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:57.543080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:57.543360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:57.544017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:57.544371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:57.545745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:57.562615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 608/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:57.564285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:57.640422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:57.679547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f99d-f4b5-45d9-a9bf-ccc7920f7791 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:57.721366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:57.722085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:57.722370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:57.722697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:57.731124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:57.732668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:32:57.745807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:57.747042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:57.748137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:57.752285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 609/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:32:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:57.753261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:57.872347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00b15-555e-4c3d-ae95-2f4eb1e3e00d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:57.900531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:57.919198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:57.920138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:57.920398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:57.920670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:57.921378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:57.921689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:57.923932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ef607-0645-4fc1-9c2a-346d5c15ec7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:57.943881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 609/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:32:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:57.944917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:58.086846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782e2c-7489-4460-82fe-555965e8fbff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:58.125946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:58.135059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:58.135671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:58.135973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:58.136212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:58.137446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=dd633df8700344a19b2de33afb3098ca) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:58.137446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:58.141960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b1d6-fca2-4e47-9090-51547dac0e0b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:58.146796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 610/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:32:57 2022] GET /identity/v3/role_assignments?user.id=dd633df8700344a19b2de33afb3098ca&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:00.683617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26598-a7a9-4703-9309-5fb00c1901d5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:00.684366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26598-a7a9-4703-9309-5fb00c1901d5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:00.684366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26598-a7a9-4703-9309-5fb00c1901d5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:00.685796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 610/1220] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:00.691234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:00.691720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:00.691816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:00.710618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:00.711954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:00.796754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 611/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:01.070341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a26598-a7a9-4703-9309-5fb00c1901d5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:01.185444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a26598-a7a9-4703-9309-5fb00c1901d5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:01.223868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:01.224500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:01.224777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:01.225049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:01.237628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:33:01.244056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:01.245124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:01.246759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 611/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:01.251900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:01.369971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a19c-b21e-45ed-a405-2293d6c61ba4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:01.398464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:01.411534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:01.412131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:01.412410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:01.412795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:01.413558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:01.413941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:01.416188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:01.431314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 612/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:01.432800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:01.487210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:01.555867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47092-7eae-4218-983a-5b5a68ed5725 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:01.591837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:01.592471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:01.592750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:01.593104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:01.604527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:01.606259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:01.623466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:01.624693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:01.625828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:01.625828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:01.625828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:01.630528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 612/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:01.631585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:01.738839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7cbe-0605-41be-bb6c-eddc8bba103d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:01.775623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:01.776226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:01.776508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:01.776785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:01.777425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:01.777732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:01.779139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:01.792333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 613/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:01.798185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:01.901110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:01.910698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e9b5f-61a3-49a2-87c0-b04bf00cdc95 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:01.956469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:01.957183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:01.957470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:01.957740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:01.967624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:01.969353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:33:01.982073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:01.983771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:01.984880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 613/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:01.989464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:02.042277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:02.112140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948851e-c3f5-4e7e-9544-793d51a4e7d1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:02.152221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:02.153348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:02.153791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:02.154069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:02.155009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:02.155383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:02.156953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:02.177380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 614/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:33:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:02.184860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:02.319215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748796b-a12d-472f-90ab-f9da9ab07845 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:02.357361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:02.358050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:02.358375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:02.358830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:02.370652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:02.372241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:02.385835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:02.391812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:02.392961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:02.392961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:02.392961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:02.392961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 614/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:02.393332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:02.510248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e874f-bd37-4071-a250-bcc2fe1b16d3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:02.513998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:02.557007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:02.557744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:02.560055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:02.560342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:02.561140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:02.561479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:02.562956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:02.575187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 615/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:02.582049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:02.593292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:02.695246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba49cd-a626-402c-8901-e32e5e65a998 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:02.737305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:02.737935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:02.738213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:02.738515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:02.746262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:02.748471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:33:02.761872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:02.762974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:02.764061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 615/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:02.769633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:02.886347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13dd8-0b41-4404-be84-6488c0beec08 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:02.943548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:02.944158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:02.944438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:02.944711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:02.945944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:02.946260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:02.947719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73f57c-d5cf-4257-893f-1f9f60155aa5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:02.962007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 616/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:33:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:02.968757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:03.043702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:03.081776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385641f-655c-4f55-8cf6-caa36d0f0b49 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:03.117264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:03.118042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:03.118146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:03.118377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/dd633df8700344a19b2de33afb3098ca/roles/14558db61f2d4609a9a37a0e28547896` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:03.135096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=dd633df8700344a19b2de33afb3098ca, role_id=14558db61f2d4609a9a37a0e28547896)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:03.136560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cdafb-c3ef-4fbe-8b03-27dc6d0f8ea9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:03.149772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 616/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:33:02 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/dd633df8700344a19b2de33afb3098ca/roles/14558db61f2d4609a9a37a0e28547896 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:33:05.491026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b8e9b-7492-4b37-a5c2-391c0c2fe49d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:05.491612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b8e9b-7492-4b37-a5c2-391c0c2fe49d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:05.491729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b8e9b-7492-4b37-a5c2-391c0c2fe49d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:05.493284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 617/1233] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:05.499554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:05.500245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:05.500245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:05.520880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:05.522376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:05.598256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 617/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:05.862586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b8e9b-7492-4b37-a5c2-391c0c2fe49d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:05.892637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:05.981648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b8e9b-7492-4b37-a5c2-391c0c2fe49d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.017961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.018613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.018892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.019163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.030575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:33:06.035327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:06.036522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:33:06.037696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:06.045621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 618/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 15:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:06.045766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:06.102312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:06.162281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a136e0-147c-4fe4-81aa-0eadc0826ad6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.201255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.201872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.202155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.202456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.203221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:06.203594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:06.205164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:06.214497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 618/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:06.220604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:06.321546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:06.340710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4084-6248-477f-9b44-3ece5d3ce913 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.375794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.376417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.376684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.376976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.387472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:06.388907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:06.404073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:06.405166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:06.406079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:06.406079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:06.406079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:06.406079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 619/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:06.411381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:06.518151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0c999-69a4-438c-a301-6e2a43547545 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.530237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:06.558310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.559008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.559264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.559522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.560157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:06.560459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:06.561888 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:06.573726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 619/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:06.579501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:06.689376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052d8b-8c42-48cb-8374-6a7f907c32c5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.730295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.730958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.731250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.731619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.734386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:06.740260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:06.742054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:33:06.753421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:06.754522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:06.755551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 620/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:33:06.761797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:06.877979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a52fee-740a-43ca-b058-91b3af25489e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:06.917552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:06.918003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:06.918384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:06.918709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:06.919523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:06.919744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:06.921328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:06.940921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 620/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:06.952896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:07.074320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3cfc7-cb3f-4611-bc26-c48f902c16f4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:07.115822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:07.116597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:07.117388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:07.117773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:07.184735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:07.192174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:07.193964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:07.306951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 621/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:33:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:33:07.313079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:07.342566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:07.420165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb6be-2607-4846-80b8-15d5bf22c479 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:07.459513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:07.460576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:07.461001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:07.461374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:07.473847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:07.475459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:07.493556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:07.494886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:07.496033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:07.496033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:07.496033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:07.496033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 621/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:07.501086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:07.607690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:07.615051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d3f6-2726-491f-896b-2c896f3afe90 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:07.652075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:07.652679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:07.652958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:07.653232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:07.653873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:07.654198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:07.655714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:07.669032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 622/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:07.674793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:07.771268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:07.801121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22317b4e-25e9-4ebe-9f9d-65eb6ad5df06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:07.842651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:07.843338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:07.843668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:07.843968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:07.852001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:07.853626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:33:07.865819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:07.872383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:07.873568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 622/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:07.875545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:07.990331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a91be-3f4c-4f14-bb7b-b8bf73595b3d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:08.031079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:08.031615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:08.031970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:08.032209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:08.033037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:08.033391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:08.034925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776b921-d2a6-4ad8-bf12-99bfd7eb0c7f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:08.048508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 623/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:08.055443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:08.167436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3faa-3112-4bc3-9cb1-af145ffa0938 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:08.206053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:08.206871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:08.207073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:08.207351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:08.208085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] RBAC: Adding query filter params (role.id=14558db61f2d4609a9a37a0e28547896, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=dd633df8700344a19b2de33afb3098ca) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:08.208463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:08.209848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a11a56-c728-413f-842a-21dd252be232 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:08.216883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 623/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:33:08 2022] GET /identity/v3/role_assignments?user.id=dd633df8700344a19b2de33afb3098ca&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=14558db61f2d4609a9a37a0e28547896 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:10.546471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3727b-6a70-46fd-97a2-44edd8548f5a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:10.546979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3727b-6a70-46fd-97a2-44edd8548f5a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:10.546979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3727b-6a70-46fd-97a2-44edd8548f5a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:10.548290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 624/1247] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:10.553861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:10.555265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:10.555395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:10.562271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:10.575062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:10.576218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:10.650016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 624/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:10.971649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3727b-6a70-46fd-97a2-44edd8548f5a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:10.993412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:11.102650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3727b-6a70-46fd-97a2-44edd8548f5a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:11.139378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:11.140005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:11.140283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:11.140555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:11.154371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:33:11.159214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:11.162607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:11.164338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 625/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:11.167267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:11.280368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79b21-40bb-410e-af2d-992ee3ee6bf2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:11.314350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:11.320583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:11.321032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:11.321309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:11.321585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:11.322286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:11.322642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:11.324196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:11.332524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 625/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:11.339466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:11.426281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:11.450486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244845b6-256d-43bd-85d0-7fa286eaac0e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:11.486712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:11.487340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:11.487618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:11.487891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:11.498701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:11.500923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:11.518143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:11.519541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:11.520681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:11.520681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:11.520681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:11.520681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 626/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:11.526312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:11.640433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487710-f95c-4a65-b10b-4c199189ca6d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:11.676348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:11.677027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:11.677361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:11.677653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:11.678355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:11.678730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:11.680948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:11.694185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 626/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:11.701201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:11.815022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d22bb-198a-4ea7-9fac-20b1882c9bbd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:11.843020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:11.852416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:11.853046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:11.853339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:11.853591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:11.861844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:11.863311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:33:11.875435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:11.876613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:11.877752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 627/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:11.886628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:11.951483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:12.002563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de627-3f41-4ca3-8e7a-d78258e457fd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:12.038311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:12.039497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:12.039778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:12.040051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:12.040798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:12.041103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:12.042593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:12.067736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 627/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:33:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:12.068858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:12.174484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316c7-f9c7-47e9-a497-7b0d0f0b931e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:12.209482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:12.210097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:12.210479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:12.210757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:12.273052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:12.286574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:12.288093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:12.391369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 628/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:33:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:33:12.397240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:12.500620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1345560-028f-471f-87fd-5ffccd139b33 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:12.515272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:12.540601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:12.541347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:12.541629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:12.541903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:12.552710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:12.554190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:12.569175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:12.570265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:12.571844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:12.571844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:12.571844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:12.571844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 628/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:12.576182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:12.664127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:12.676516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008c050-fd83-41cf-b9c5-f6273250aa21 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:12.710467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:12.711633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:12.711847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:12.712118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:12.712749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:12.713050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:12.714436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:12.725238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 629/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:12.731010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:12.828342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2847a1a-ba10-4ef0-95b1-492928a119fd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:12.860402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:12.861016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:12.861298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:12.861574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:12.868004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:12.869441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:33:12.881181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:12.882750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:12.883678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 629/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:12.895466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:13.002632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8622d9f-6722-4afb-b72f-49863961e140 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:13.035550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:13.036143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:13.036427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:13.036703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:13.037391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:13.037697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:13.039165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e43e-f62b-485d-ab56-3a4754460daa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:13.050671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 630/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:33:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:13.057602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:13.083185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:13.154717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76e36-c8de-4615-9704-543570ae500a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:13.187039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:13.187637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:13.187920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:13.188193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:13.188914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=dd633df8700344a19b2de33afb3098ca) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:13.189221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:13.190660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b280cc-d9c3-4de3-b4cd-e8dd4037e7d4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:13.196424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 630/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:33:13 2022] GET /identity/v3/role_assignments?user.id=dd633df8700344a19b2de33afb3098ca&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:15.301413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece9c6-0716-40d1-a156-5379927c7aba None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:15.301907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece9c6-0716-40d1-a156-5379927c7aba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:15.302036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece9c6-0716-40d1-a156-5379927c7aba None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:15.303339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 631/1261] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:15.309340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268197c-021e-475b-a447-468a882fb3aa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:15.309732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268197c-021e-475b-a447-468a882fb3aa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:15.309891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268197c-021e-475b-a447-468a882fb3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:15.327578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0268197c-021e-475b-a447-468a882fb3aa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:15.328783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0268197c-021e-475b-a447-468a882fb3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:15.393135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 631/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:15.665748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eece9c6-0716-40d1-a156-5379927c7aba None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:15.676426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:15.763437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eece9c6-0716-40d1-a156-5379927c7aba None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:15.796123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:15.796735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:15.797017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:15.797291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:15.808013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:33:15.812768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:15.813882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:15.814980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 632/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:33:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:15.820379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268197c-021e-475b-a447-468a882fb3aa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:15.838528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:15.922331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268197c-021e-475b-a447-468a882fb3aa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:15.966256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:15.966941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:15.967177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:15.967488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:15.968149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:15.968446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:15.969798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:15.977985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 632/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:33:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:15.983912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.048484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:16.081552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0db1e-187f-4c2c-81a3-5dce178a4c8f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.113437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.114059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.114336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.114643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.124925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.126373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:16.140973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:16.142119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:16.143080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:16.143080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:16.143080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:16.143080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 633/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:33:16.147838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.226845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:16.245785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ef42-641b-41d5-8463-c547c456ccd1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.278993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.279599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.279883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.280161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.280802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:16.281109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.282505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.293755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 633/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:16.299406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.396361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc336ef-e5bb-41f3-b27a-f3b78be80cc6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.416387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:16.430349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.430995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.431271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.431611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.438783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.440201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:33:16.451746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:16.452859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:16.454083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 634/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:16.458812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.555102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021bb56-ce92-43df-9e9f-1ee6332e7d47 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.588177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.588775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.589055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.589330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.590046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:16.590352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.591788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.600116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:16.609894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 634/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:16.615728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.712921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f336f2-2b83-474b-8ce8-b6d31979f089 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.744197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.744808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.745089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.745360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.755273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.756712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:16.770949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:16.772096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:16.773308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:16.773308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:16.773308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:16.773308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 635/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:16.778553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.875649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365d31-9f6e-4a1d-94d9-fa3f9bc5ff00 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:16.915609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:16.916223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:16.916500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:16.916789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:16.917424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:16.917727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:16.919120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:16.931455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 635/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:16.937159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:16.969059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:17.035317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b972b-35e2-4ed3-97d8-2624e64722d9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:17.067768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:17.068390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:17.068671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:17.068945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:17.075296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:17.076731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:33:17.088263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:17.089303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:17.090596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 636/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:17.095190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:17.151112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:17.191944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a881a-1f67-4d14-8bcf-1eb0a260d7c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:17.224578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:17.225174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:17.225454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:17.225852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:17.227046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:17.227388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:17.228809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e996fc-61d6-4eb1-b4ab-b0f79fff5afa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:17.240731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 636/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:17.246769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:17.334626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:17.343998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbd07-ca3b-4114-a754-49a07b24d1b9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:17.375179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:17.375832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:17.376107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:17.376377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] PATH_INFO: `/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/dd633df8700344a19b2de33afb3098ca/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:17.378215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] RBAC: Authorizing `identity:create_grant(project_id=f0522609f3e2487d8abf38206ef39d4e, user_id=dd633df8700344a19b2de33afb3098ca, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:17.379643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdca8b-b287-433b-b801-732b17c61b0b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:17.392507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 637/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 15:33:17 2022] PUT /identity/v3/projects/f0522609f3e2487d8abf38206ef39d4e/users/dd633df8700344a19b2de33afb3098ca/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:33:19.575214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32391d0e-2988-4958-8aa6-a0ad6842c9a7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:19.575750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32391d0e-2988-4958-8aa6-a0ad6842c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:19.575750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32391d0e-2988-4958-8aa6-a0ad6842c9a7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:19.577004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 637/1274] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:19.582235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:19.582675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:19.582791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:19.601126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:19.602299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:19.668016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 638/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:19.952604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32391d0e-2988-4958-8aa6-a0ad6842c9a7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.053995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32391d0e-2988-4958-8aa6-a0ad6842c9a7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.070580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:20.087457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.088069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.088342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.088608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.098187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:33:20.103516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:20.104662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:20.105912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 638/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 15:33:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:33:20.110571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.189579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:20.208772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b2f4-99ea-4906-93f1-33a2011556b9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.241730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.242628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.242909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.243181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.243837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:20.244143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:20.245524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:20.253793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 639/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:20.260153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.355486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a198-2290-48df-a739-77a4499c8a14 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.388417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.389040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.389315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.389583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.400127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:20.401553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:20.418601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:20.420521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:20.421646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:20.421646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:20.421646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:20.421646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 639/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:20.428157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.526606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1faac-15ff-46cf-bde4-1d2a2ea24484 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.559717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.560316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.560592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.560863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.561492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:20.561790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:20.563169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:20.575100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 640/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:20.580824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.611156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:20.678379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fb83-219b-4ce8-ba82-183c59e65fb1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.726156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.726952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.727237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.727513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.737005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:20.738465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:33:20.750207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:20.751377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:20.752762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 640/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:20.757841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:20.761629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:20.853077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d44bb-6302-49bb-8b72-449007c63aaa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:20.886571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:20.887182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:20.887455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:20.887725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:20.888454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:20.888760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:20.890189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:20.908951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 641/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:20.914803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.002188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:21.011614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c2901-dcef-4911-ab11-c00453cef1d6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.043590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.044186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.044467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.044739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.108837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.110200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.195193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:21.205312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 641/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 15:33:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:33:21.211356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.270370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:21.308815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38150923-7fdc-4f7e-b359-bd68c295bc94 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.341728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.342351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.342657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.342929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.353086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.354565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:33:21.368799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:21.369738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:33:21.370719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:33:21.370719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:33:21.370719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:21.370719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 642/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:21.375613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.470599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51e0e5-acfa-477d-aba6-de819164c8d6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.503433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.504028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.504307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.504585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.505260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:21.505570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.507026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.517920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 642/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 15:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:21.524059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.623421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6b30-cdfb-4d1e-a0de-c0daf9630b07 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.636877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:21.657412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.658035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.658317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.658628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.665632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.667108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:33:21.677932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:21.679036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:33:21.680439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 643/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 15:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:21.685113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.733762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:21.782915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601ef1-cf4f-4cac-87bb-05d6258a78e8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.814631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.815226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.815506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.815779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.816464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:21.816769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.818178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d0590-f5f8-4ac8-b5d0-3e210f6ac9e0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.830723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 643/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 15:33:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:21.837372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:21.936070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c9c9-901f-4244-b955-ed74d7c52f6e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:21.968951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:21.969549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:21.969824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:21.970096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:21.970857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] RBAC: Adding query filter params (role.id=161e292b353449f4bc2d222d2312b3c5, scope.project.id=f0522609f3e2487d8abf38206ef39d4e, user.id=dd633df8700344a19b2de33afb3098ca) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:21.971163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:21.972580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285093-1519-4d99-bfbd-ec091763ac73 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:21.978189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 644/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 15:33:21 2022] GET /identity/v3/role_assignments?user.id=dd633df8700344a19b2de33afb3098ca&scope.project.id=f0522609f3e2487d8abf38206ef39d4e&role.id=161e292b353449f4bc2d222d2312b3c5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:24.139234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34d0c-0154-4d2e-8a70-b13160a47117 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:24.139769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34d0c-0154-4d2e-8a70-b13160a47117 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:24.139769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34d0c-0154-4d2e-8a70-b13160a47117 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:24.141108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 644/1288] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:24.146426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:24.146854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:24.147142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:24.165995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:24.167214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:24.196491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:24.232778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 645/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:24.502066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34d0c-0154-4d2e-8a70-b13160a47117 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:24.599618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34d0c-0154-4d2e-8a70-b13160a47117 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:24.610407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:24.632789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:24.633407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:24.633686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:24.633958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:24.634577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:24.635931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:24.644558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:24.645766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:24.646660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 645/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:33:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:24.652297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:24.747585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232525c1-82c7-4a3e-a50f-e4bee0aa7483 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:24.780632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:24.781158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:24.781439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:24.781712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:24.782329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:24.782664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:24.784021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8871a44-610a-428a-9daa-63a5691ad0f4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:24.788796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 646/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 15:33:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:24.795300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:24.906345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d87d13-0e32-4662-8ca9-20c0df7248a4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:24.948743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:24.949588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:24.949935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:24.950241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:24.951061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:24.951396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:24.952958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ba752-e00a-4112-b0e4-b5c4472762c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:24.958312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 646/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 15:33:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:27.116747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d33c-da6c-4596-a174-55eef47d2995 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:27.117307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d33c-da6c-4596-a174-55eef47d2995 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:27.117307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d33c-da6c-4596-a174-55eef47d2995 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:27.118670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 647/1293] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:27.123682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8a92d-1fdd-4805-9565-a923e3c816f4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:27.124092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8a92d-1fdd-4805-9565-a923e3c816f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:27.124241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8a92d-1fdd-4805-9565-a923e3c816f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:27.142573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3dc8a92d-1fdd-4805-9565-a923e3c816f4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:27.143757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3dc8a92d-1fdd-4805-9565-a923e3c816f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:27.172814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:27.210470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 647/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:27.477306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d33c-da6c-4596-a174-55eef47d2995 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:27.508293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:27.577853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d33c-da6c-4596-a174-55eef47d2995 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:27.609971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:27.610585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:27.610863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:27.611130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:27.611777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:27.613118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b25b2-3c70-4b7c-9927-229bdf70fbf1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:27.627526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 648/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 15:33:27 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:33:29.800526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be12464-61a7-4d20-9bc3-a5d5ea26dcd9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:29.801118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be12464-61a7-4d20-9bc3-a5d5ea26dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:29.801118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be12464-61a7-4d20-9bc3-a5d5ea26dcd9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:29.802403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 648/1296] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:29.807920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:29.808373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:29.808475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:29.826155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:29.827346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:29.898847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 649/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:33:30.172314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be12464-61a7-4d20-9bc3-a5d5ea26dcd9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:30.270846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be12464-61a7-4d20-9bc3-a5d5ea26dcd9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:30.304601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:30.305222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:30.305498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:30.305765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:30.306341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:30.307877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:30.316429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:30.317650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:30.319227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 649/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:33:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:30.324133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:30.341788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:30.422061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf4d5-6185-4d0b-81d6-7c314021fdc7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:30.455938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:30.456462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:30.456740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:30.457009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:30.457633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:30.457937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:30.459336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:30.464369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 650/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 15:33:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:30.470390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:30.500839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:30.573718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a80ec-4824-4330-93ef-5de3e8629bb5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:30.605004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:30.605597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:30.605873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:30.606141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:30.606871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] RBAC: Adding query filter params (interface=public, service_id=6e868c21353545ae9543b03b100113fa, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:30.607176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:30.608515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d216ac5-aaab-48dc-8754-695c6e352f5d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:30.613242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 650/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 15:33:30 2022] GET /identity/v3/endpoints?service_id=6e868c21353545ae9543b03b100113fa&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:33:30.619470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:30.701834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:30.720083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9d62-680e-4fcc-b1a9-24842f81cb48 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:30.753766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:30.754361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:30.754667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:30.754935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:30.755532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:30.756886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb963cf4-ea17-4b6e-8f8a-f3118d3de26b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:30.763532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 651/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 15:33:30 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:33:32.880040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca1c9-84fd-4125-8cb9-1b9caf174428 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:32.880580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca1c9-84fd-4125-8cb9-1b9caf174428 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:32.880580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca1c9-84fd-4125-8cb9-1b9caf174428 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:32.881926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 651/1302] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:32.889916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:32.890429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:32.890528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:32.908664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:32.909872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:32.975627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 652/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:33:33.246604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaca1c9-84fd-4125-8cb9-1b9caf174428 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:33.339285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:33.346190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaca1c9-84fd-4125-8cb9-1b9caf174428 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:33.377955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:33.378596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:33.378872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:33.379141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:33.379712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:33.381608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:33:33.390022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:33:33.391211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:33:33.392432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 652/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 15:33:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:33:33.397672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:33.475652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:33.497612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec906bf3-64d9-4874-8b47-e1af14788048 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:33.529981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:33.530597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:33.530876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:33.531146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:33.531771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:33:33.532070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:33.533418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c25c4-8b44-4565-87d8-a13260fc8f01 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:33.538982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 653/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 15:33:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:33:33.545198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:33.641977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2617-1745-4383-af7e-72af240688d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:33.675641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:33.676237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:33.676511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:33.676781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:33.677435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:33.678875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e178bb7-4f17-446d-af2a-f4bf581cdb73 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:33.697827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 653/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:33:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:33:43.535549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=30 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:43.693579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:38.864717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a2f0-002d-424f-90d5-d1f35ad4085d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:38.865292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a2f0-002d-424f-90d5-d1f35ad4085d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:38.865292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a2f0-002d-424f-90d5-d1f35ad4085d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:38.866651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 654/1307] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:38.873476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03f60-eac6-4a13-9a1f-986bd73c62c9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:38.877002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03f60-eac6-4a13-9a1f-986bd73c62c9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:38.879574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03f60-eac6-4a13-9a1f-986bd73c62c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:38.986363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5fc03f60-eac6-4a13-9a1f-986bd73c62c9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:38.987745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fc03f60-eac6-4a13-9a1f-986bd73c62c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:39.099884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 654/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:39.401683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:39.402151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:39.402442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:39.403797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 655/1309] 10.222.0.60 () {54 vars in 964 bytes} [Wed Sep 28 15:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:39.409872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc353b-a5e6-4aed-ad5d-a08a15910cce None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:39.410233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc353b-a5e6-4aed-ad5d-a08a15910cce None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:39.410620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc353b-a5e6-4aed-ad5d-a08a15910cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:39.498715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6afc353b-a5e6-4aed-ad5d-a08a15910cce None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:39.500032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6afc353b-a5e6-4aed-ad5d-a08a15910cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:39.522897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:39.585124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 655/1310] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Sep 28 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:39.594376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:39.596177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:39.709583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feafdb8-414b-4d4f-954f-108f5b70d6d3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:39.787723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:39.788495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:39.788872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:39.789240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:39.790968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:39.849753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:39.911383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:39.912996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb382-8c96-40ef-afa6-712621fb2494 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:39.937213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 656/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:42.470974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:42.471502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:42.471502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:42.474582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 656/1312] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:42.479432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5fe9b-9332-413f-a081-e7cfa8e170cf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:42.479887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5fe9b-9332-413f-a081-e7cfa8e170cf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:42.479992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5fe9b-9332-413f-a081-e7cfa8e170cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:42.551449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d0b5fe9b-9332-413f-a081-e7cfa8e170cf None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:42.552823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d0b5fe9b-9332-413f-a081-e7cfa8e170cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:42.653531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 657/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:42.957124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:42.958736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:43.061144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ed6e6-61d0-445d-9256-61f0d60b9626 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:43.139308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:43.141706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:43.142374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:43.142902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:43.143408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:43.145479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:43.253961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:43.255419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4387719-17c8-4377-ae8c-fead1f6d289f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:43.273633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 657/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:45.747144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:45.747682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:45.747682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:45.749032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 658/1315] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:45.754461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e408a6-649e-4b92-bc94-f34875b5b528 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:45.754907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e408a6-649e-4b92-bc94-f34875b5b528 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:45.755004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e408a6-649e-4b92-bc94-f34875b5b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:45.823164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73e408a6-649e-4b92-bc94-f34875b5b528 None None] MFA Rules not processed for user `ed4d4992ad9c4cb6b2c992b66fa6cd70`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:45.824358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73e408a6-649e-4b92-bc94-f34875b5b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:45.874824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:45.920357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 658/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:34:46.201970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:46.203733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:46.214607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:46.308447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67bcc1-c25d-4dd2-b654-fd70a1e61a1f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:46.379960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:46.380559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:46.380839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:46.381112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:46.382742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:46.444406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:46.490882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:46.492273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39e41-d0bd-42aa-87d1-089aa22a0fc0 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:46.510144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 659/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:49.229565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f5be-4d4b-4d6b-b38d-34a8ab6a5c61 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.230107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f5be-4d4b-4d6b-b38d-34a8ab6a5c61 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.230107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f5be-4d4b-4d6b-b38d-34a8ab6a5c61 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.231489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 659/1318] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:49.236658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89e71a-836d-4fef-b9f5-9ff7f5cf15ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.237104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89e71a-836d-4fef-b9f5-9ff7f5cf15ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.237243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89e71a-836d-4fef-b9f5-9ff7f5cf15ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.301723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8b89e71a-836d-4fef-b9f5-9ff7f5cf15ee None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:49.303211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8b89e71a-836d-4fef-b9f5-9ff7f5cf15ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:49.398257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 660/1319] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:49.705749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.706025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.706300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.707705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 660/1320] 10.222.0.60 () {54 vars in 964 bytes} [Wed Sep 28 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:49.713520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1de7ac-bd52-4fa3-809b-85865c900044 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.713806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1de7ac-bd52-4fa3-809b-85865c900044 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.714084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1de7ac-bd52-4fa3-809b-85865c900044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.735732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:49.789350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5f1de7ac-bd52-4fa3-809b-85865c900044 None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:49.790593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5f1de7ac-bd52-4fa3-809b-85865c900044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:49.867419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 661/1321] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Sep 28 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:49.877232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:49.878927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:49.983247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347c23b-6769-44cb-a618-d797039524c2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:50.054838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:50.055428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:50.055702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:50.055964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:50.057556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:50.064830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:50.162209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:50.163661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a009532-0713-47a6-9efe-607f0ebb7f45 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:50.181831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 661/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:53.499271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.499800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.499913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.501741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 662/1323] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:53.507258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a99c1-0517-4a38-8538-b0b83824fe36 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.507706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a99c1-0517-4a38-8538-b0b83824fe36 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.507961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a99c1-0517-4a38-8538-b0b83824fe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.580956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fd7a99c1-0517-4a38-8538-b0b83824fe36 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:53.582074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd7a99c1-0517-4a38-8538-b0b83824fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.645239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:53.688308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 662/1324] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:53.971580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:53.973215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:54.077983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54f5e4-f678-4bf3-8bdb-376e5fbe690d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:54.090743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:54.152613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:54.153216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:54.153494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:54.153765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:54.155415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:54.270952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:54.272398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d2de-6e67-4b59-b9ce-1986b70f0a18 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:54.290139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 663/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:56.528214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:56.528733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:56.528839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:56.530302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 663/1326] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:56.535600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2530ac-14cf-4b06-8af8-4dadb04da06b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:56.536166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2530ac-14cf-4b06-8af8-4dadb04da06b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:56.536166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2530ac-14cf-4b06-8af8-4dadb04da06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:56.584621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:56.603129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cd2530ac-14cf-4b06-8af8-4dadb04da06b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:56.604340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cd2530ac-14cf-4b06-8af8-4dadb04da06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:56.710805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 664/1327] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:56.998524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:57.000046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:57.107716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1811c-194a-4a69-a7da-ebeda7965bd3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:57.192430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:57.193162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:57.193518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:57.193853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:57.195613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:57.214261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:57.314972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:57.316346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1e3d6-f006-41a3-9fce-1457016f0412 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:57.334508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 664/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:59.596217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:59.596720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:59.596720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:59.598026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 665/1329] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:59.603382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1565bf-97f3-4b2a-b66a-05cc284e86fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:59.603858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1565bf-97f3-4b2a-b66a-05cc284e86fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:59.603964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1565bf-97f3-4b2a-b66a-05cc284e86fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:59.671691 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef1565bf-97f3-4b2a-b66a-05cc284e86fc None None] MFA Rules not processed for user `ed4d4992ad9c4cb6b2c992b66fa6cd70`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:59.672854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef1565bf-97f3-4b2a-b66a-05cc284e86fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:59.739285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:59.770761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 665/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:00.056349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:00.057944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:00.171298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f1e20-1bb8-443c-85da-d83c093f9f72 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:00.194845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:00.263521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:00.264249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:00.264918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:00.265000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:00.267342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:00.395306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:00.396887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75b305-2211-44fd-8f03-4a89eb4a7192 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:00.416334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 666/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:02.815639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631a32-26ae-44ec-adf6-97929091807e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:02.816235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631a32-26ae-44ec-adf6-97929091807e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:02.816235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631a32-26ae-44ec-adf6-97929091807e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:02.817500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 666/1332] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:02.823627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1304-1f3c-43d2-a5dd-c9196f5b22a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:02.824078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1304-1f3c-43d2-a5dd-c9196f5b22a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:02.824261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1304-1f3c-43d2-a5dd-c9196f5b22a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:02.894379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:02.898037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e26e1304-1f3c-43d2-a5dd-c9196f5b22a4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:02.899747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e26e1304-1f3c-43d2-a5dd-c9196f5b22a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:03.007329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 667/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:03.291113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631a32-26ae-44ec-adf6-97929091807e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:03.292714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35631a32-26ae-44ec-adf6-97929091807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:03.397287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631a32-26ae-44ec-adf6-97929091807e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:03.468828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:03.470010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:03.470294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:03.470594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:03.472236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:03.498141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:03.582731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:03.584090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fd138-75a5-4cc3-acca-e27e2abfdc14 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:03.602717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 667/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:05.811372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:05.811916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:05.811916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:05.813791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 668/1335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:05.819529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d40a-6ce3-40d3-bc68-71fb6ddda745 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:05.819941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d40a-6ce3-40d3-bc68-71fb6ddda745 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:05.820069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d40a-6ce3-40d3-bc68-71fb6ddda745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:05.885090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c610d40a-6ce3-40d3-bc68-71fb6ddda745 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:05.886319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c610d40a-6ce3-40d3-bc68-71fb6ddda745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:05.965392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:05.989830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 668/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:06.286557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:06.288123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:06.401492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4306ba3-6ecf-4364-8966-c71c47aa7e35 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:06.430607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:06.475891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:06.476497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:06.476777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:06.477048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:06.478694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:06.592177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:06.593520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eec5c-798c-4b5a-9be7-846aad028daa service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:06.611832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 669/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:09.176686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:09.177492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:09.177492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:09.178838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 669/1338] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:09.184061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cf501-d9b2-49cc-805f-929db5b27987 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:09.184575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cf501-d9b2-49cc-805f-929db5b27987 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:09.184699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cf501-d9b2-49cc-805f-929db5b27987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:09.249905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-657cf501-d9b2-49cc-805f-929db5b27987 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:09.251160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-657cf501-d9b2-49cc-805f-929db5b27987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:09.259639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:09.347503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 670/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:35:09.629679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:09.631295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:09.733296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1694b-7f75-46a2-8a5a-ecc2e9713d07 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:09.804780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:09.805379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:09.805656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:09.805925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:09.807539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:09.842170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:09.916271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:09.917763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88d396-4632-4a35-92a6-728d4336347e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:09.936121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 670/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:14.068858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:14.069364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:14.069445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:14.074906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 671/1341] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:14.076555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308244b8-6085-4f9f-928b-bbf516c1774c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:14.077357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308244b8-6085-4f9f-928b-bbf516c1774c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:14.078391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308244b8-6085-4f9f-928b-bbf516c1774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:14.163773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-308244b8-6085-4f9f-928b-bbf516c1774c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:14.164927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-308244b8-6085-4f9f-928b-bbf516c1774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:14.273443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:14.285266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 671/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:14.567500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:14.569030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:14.678403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b5a6f-7629-4371-bdb2-01c4733126bb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:14.716393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:14.753333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:14.753946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:14.754227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:14.754535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:14.756161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:14.864886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:14.866233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a165d9f-2ff2-4265-a33b-ad550557f26f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:14.887613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 672/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:17.650672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:17.651199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:17.651313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:17.652571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 672/1344] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:17.658661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43068-1cd7-4ccc-9374-88783ca2688e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:17.659084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43068-1cd7-4ccc-9374-88783ca2688e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:17.659329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43068-1cd7-4ccc-9374-88783ca2688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:17.725088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-afa43068-1cd7-4ccc-9374-88783ca2688e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:17.726295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-afa43068-1cd7-4ccc-9374-88783ca2688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:17.743868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:17.822605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 673/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:18.109258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:18.110814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:18.217806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46e061-762a-40ad-8811-0b3d286438f0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:18.299024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:18.299649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:18.299928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:18.300201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:18.302153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:18.348378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:18.415511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:18.417403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98418e90-3414-4bd5-8b73-1dd13806fbc0 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:18.437800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 673/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:22.704879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:22.705363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:22.705555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:22.706981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 674/1347] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:22.712838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e930-e8a2-494d-b569-77bfbe5b463d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:22.713271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e930-e8a2-494d-b569-77bfbe5b463d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:22.713388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e930-e8a2-494d-b569-77bfbe5b463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:22.785590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f2b8e930-e8a2-494d-b569-77bfbe5b463d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:22.786845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2b8e930-e8a2-494d-b569-77bfbe5b463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:22.879886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:22.882101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 674/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:23.181671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:23.183285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:23.286989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f9c6a-24f3-4774-a970-1ff18aefd2b0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:23.335683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:23.363994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:23.364601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:23.364880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:23.365150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:23.366859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:23.473468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:23.474853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57323e-ee1a-4b46-8c66-067cf5921f93 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:23.493296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 675/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:28.479688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dbe3-4ce1-4d16-ac84-d64fb0bef564 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:28.480190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dbe3-4ce1-4d16-ac84-d64fb0bef564 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:28.480190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dbe3-4ce1-4d16-ac84-d64fb0bef564 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:28.482907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 675/1350] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:28.487553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227371f-05fe-4fca-8e92-19ab896bbe29 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:28.487982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227371f-05fe-4fca-8e92-19ab896bbe29 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:28.488159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227371f-05fe-4fca-8e92-19ab896bbe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:28.624768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2227371f-05fe-4fca-8e92-19ab896bbe29 None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:28.626149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2227371f-05fe-4fca-8e92-19ab896bbe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:28.700554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:28.830853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 676/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:32.870947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6aab2-f2b9-4b54-be16-4bfa46df9bbe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:32.871484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6aab2-f2b9-4b54-be16-4bfa46df9bbe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:32.871484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6aab2-f2b9-4b54-be16-4bfa46df9bbe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:32.872835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 676/1352] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:32.878197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141203-a5c4-457c-a315-79808529ef6c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:32.879110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141203-a5c4-457c-a315-79808529ef6c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:32.879574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141203-a5c4-457c-a315-79808529ef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:32.964392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-32141203-a5c4-457c-a315-79808529ef6c None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:32.966131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32141203-a5c4-457c-a315-79808529ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:33.052740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:33.093594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 677/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:34.705067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc4b8-4a3d-4243-aa43-b28004228750 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:34.705602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc4b8-4a3d-4243-aa43-b28004228750 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:34.705832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc4b8-4a3d-4243-aa43-b28004228750 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:34.708100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 677/1354] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:35:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:34.714182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfdce-c7f0-4543-a40c-be42cfbf1cfb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:34.714696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfdce-c7f0-4543-a40c-be42cfbf1cfb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:34.714902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfdce-c7f0-4543-a40c-be42cfbf1cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:34.791503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2b4bfdce-c7f0-4543-a40c-be42cfbf1cfb None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:34.792707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2b4bfdce-c7f0-4543-a40c-be42cfbf1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:34.899923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 678/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:38.525431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532a90-2f95-4478-9b0f-c0f9530230de None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:38.525986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532a90-2f95-4478-9b0f-c0f9530230de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:38.526440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532a90-2f95-4478-9b0f-c0f9530230de None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:38.528113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 678/1356] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:38.534536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7cde-935d-4dee-89ab-7750e16f2ac2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:38.535401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7cde-935d-4dee-89ab-7750e16f2ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:38.535836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7cde-935d-4dee-89ab-7750e16f2ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:38.585910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:38.618289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-917e7cde-935d-4dee-89ab-7750e16f2ac2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:38.619526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-917e7cde-935d-4dee-89ab-7750e16f2ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:38.728544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 679/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:39.039808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:39.040318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:39.040773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:39.043203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 679/1358] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:35:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:39.054034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15f2b8-3f63-4b7c-adce-d801e9a37b5f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:39.054656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15f2b8-3f63-4b7c-adce-d801e9a37b5f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:39.055142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15f2b8-3f63-4b7c-adce-d801e9a37b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:39.142559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ea15f2b8-3f63-4b7c-adce-d801e9a37b5f None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:39.143994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ea15f2b8-3f63-4b7c-adce-d801e9a37b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:39.194469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:39.276934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 680/1359] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:39.285126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:39.286833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:39.451936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e994-596c-49b5-94a7-b9f140938a37 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:39.528810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:39.570895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:39.571503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:39.571788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:39.572062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:39.573723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:39.666960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:39.668740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4463529-c11c-449a-8f68-bbd6fa494874 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:39.689675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 680/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:42.088905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:42.089420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:42.089535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:42.090995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 681/1361] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:42.096837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2923-f402-4b51-a3f0-6fabb229d433 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:42.097338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2923-f402-4b51-a3f0-6fabb229d433 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:42.097657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2923-f402-4b51-a3f0-6fabb229d433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:42.171989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-096e2923-f402-4b51-a3f0-6fabb229d433 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:42.177642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-096e2923-f402-4b51-a3f0-6fabb229d433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:42.252472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:42.322014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 681/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:42.616466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:42.618035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:42.800664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adca0a-71d4-487f-8f2c-72ae20203a81 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:42.816358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:42.958730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:42.958730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:42.959263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:42.959263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:42.961294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:43.057098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:43.058532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9173-4ce3-47d2-9744-800ca6dabe63 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:43.077943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 682/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:45.586366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197fdd-4a1a-450d-a0b7-71cb61140fe6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:45.586993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197fdd-4a1a-450d-a0b7-71cb61140fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:45.586993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197fdd-4a1a-450d-a0b7-71cb61140fe6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:45.588446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 682/1364] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:45.593793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32bef-0908-4409-a398-3e7644619081 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:45.594288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32bef-0908-4409-a398-3e7644619081 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:45.594381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32bef-0908-4409-a398-3e7644619081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:45.611172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:45.689790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c5a32bef-0908-4409-a398-3e7644619081 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:45.691467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c5a32bef-0908-4409-a398-3e7644619081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:45.803238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 683/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:46.118940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.119373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.119750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.121224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 683/1366] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:46.127051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5aa0-3c8c-4c36-b584-a27dd50f68f0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.127350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5aa0-3c8c-4c36-b584-a27dd50f68f0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.127607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5aa0-3c8c-4c36-b584-a27dd50f68f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.209015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cbee5aa0-3c8c-4c36-b584-a27dd50f68f0 None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:46.210217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cbee5aa0-3c8c-4c36-b584-a27dd50f68f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:46.252780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:46.340722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 684/1367] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:46.349170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:46.350701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:46.508149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5e78a-55db-43f2-aa61-d2305f814296 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:46.530142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:46.630208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:46.630951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.631210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.631450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.633001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:46.732906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:46.734556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d236e03-2509-4869-91ab-25812dfbfcb7 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:46.754548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 684/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:46.911864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9635a0-2cb6-4357-9bd3-b4079ea3da46 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.912451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9635a0-2cb6-4357-9bd3-b4079ea3da46 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.912563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9635a0-2cb6-4357-9bd3-b4079ea3da46 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.914194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 685/1369] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:46.920251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba087d-2092-4b7d-82e6-6ac0782e5a1d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.920574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba087d-2092-4b7d-82e6-6ac0782e5a1d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.920809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba087d-2092-4b7d-82e6-6ac0782e5a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.958002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:47.000635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5cba087d-2092-4b7d-82e6-6ac0782e5a1d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:47.001891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5cba087d-2092-4b7d-82e6-6ac0782e5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:47.118488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 685/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:47.455131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:47.455131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:47.455131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:47.456395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 686/1371] 10.222.0.60 () {54 vars in 963 bytes} [Wed Sep 28 15:35:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:47.461636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798bd3-2945-4501-bc1f-d25aedc815cc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:47.462123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798bd3-2945-4501-bc1f-d25aedc815cc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:47.462406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798bd3-2945-4501-bc1f-d25aedc815cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:47.564071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c2798bd3-2945-4501-bc1f-d25aedc815cc None None] MFA Rules not processed for user `9685e7e865604d4b9acd36902045d141`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:47.565295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2798bd3-2945-4501-bc1f-d25aedc815cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:47.617837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:47.650606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 686/1372] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Sep 28 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:47.658434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:47.661192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:47.788741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb7413-889f-4543-b6cf-d266bf95d9ca None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:47.838021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:47.882292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:47.882292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:47.882292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:47.882292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:47.886067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:48.091682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:48.093291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedf2a1-ee08-4c9e-b964-742ae349c4cd service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:48.118752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 687/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:35:49.024479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:49.024993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:49.024993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:49.026256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 687/1374] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:49.031601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77130c8-1077-4a2c-8666-9b25b29dd8b5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:49.031883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77130c8-1077-4a2c-8666-9b25b29dd8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:49.032153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77130c8-1077-4a2c-8666-9b25b29dd8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:49.098332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f77130c8-1077-4a2c-8666-9b25b29dd8b5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:49.099557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f77130c8-1077-4a2c-8666-9b25b29dd8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:49.120167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:49.197559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 688/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:49.486865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:49.488616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:49.657570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cdbef-7580-4eb7-aabd-738fa38c3dbe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:49.715186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:49.785187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:49.785814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:49.786100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:49.786375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:49.788145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:49.885860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:49.887805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355643-3a79-49a9-9666-59cc1b329200 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:49.906406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 688/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:52.192312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:52.192878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:52.192878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:52.194354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 689/1377] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:52.199860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86389152-ba26-42d5-b5bc-8936173527bb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:52.201254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86389152-ba26-42d5-b5bc-8936173527bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:52.201789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86389152-ba26-42d5-b5bc-8936173527bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:52.277836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-86389152-ba26-42d5-b5bc-8936173527bb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:52.279665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86389152-ba26-42d5-b5bc-8936173527bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:52.287499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:52.393625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 689/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:52.687363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:52.689016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:52.842844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323cba9-7179-4a32-a78b-2eba3f91a716 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:52.847333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:52.978029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:52.978645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:52.978933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:52.979199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:52.980818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:53.107392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:53.109150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606909-3f4b-43ea-aab1-d50b76c55038 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:53.128057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 690/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:55.393801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:55.394538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:55.394538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:55.396067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 690/1380] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:55.401330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36112-aea6-4714-8d51-564bd679faca None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:55.401737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36112-aea6-4714-8d51-564bd679faca None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:55.401909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36112-aea6-4714-8d51-564bd679faca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:55.417230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:55.476546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-47d36112-aea6-4714-8d51-564bd679faca None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:55.477769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-47d36112-aea6-4714-8d51-564bd679faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:55.587256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 691/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:55.889181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:55.890744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:56.045208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:56.057481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799153c-8c06-4771-b9c1-3dc5800b346c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:56.203636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:56.204290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:56.204649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:56.204986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:56.206944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:56.330094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:56.331665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f9ffe-cfc6-4da5-8b8a-f6baf7ba6ed8 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:56.341711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:56.353029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 691/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:35:59.129986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:59.130827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:59.130827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:59.132531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 692/1383] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:59.139023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830dbb-6908-4453-ac9b-f09ef7a417c2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:59.139578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830dbb-6908-4453-ac9b-f09ef7a417c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:59.139578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830dbb-6908-4453-ac9b-f09ef7a417c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:59.218497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4d830dbb-6908-4453-ac9b-f09ef7a417c2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:59.219702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d830dbb-6908-4453-ac9b-f09ef7a417c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:59.366742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 692/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:59.664521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:59.666075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:59.774248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:59.841774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a661e-9676-4a8d-908d-5155d43b0ea8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:00.075083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:00.075714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:00.075986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:00.076251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:00.077836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:00.141164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:00.189279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:00.190931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e3ee7-359f-4035-9b7d-719a28174c9a service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:00.209268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 693/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:02.665539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16026307-41ec-48aa-9344-4f0243301908 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:02.666081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16026307-41ec-48aa-9344-4f0243301908 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:02.666081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16026307-41ec-48aa-9344-4f0243301908 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:02.667519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 693/1386] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:02.673967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2afd-95d7-4542-8c7b-73f6b1c0fd9f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:02.674486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2afd-95d7-4542-8c7b-73f6b1c0fd9f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:02.674692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2afd-95d7-4542-8c7b-73f6b1c0fd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:02.753234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-deec2afd-95d7-4542-8c7b-73f6b1c0fd9f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:02.754452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-deec2afd-95d7-4542-8c7b-73f6b1c0fd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:02.868673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 694/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:03.179054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16026307-41ec-48aa-9344-4f0243301908 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:03.180578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16026307-41ec-48aa-9344-4f0243301908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:03.261237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:03.332395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16026307-41ec-48aa-9344-4f0243301908 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:03.445027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:03.445553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:03.445833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:03.446109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:03.447753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:03.488479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:03.546110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:03.547746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cae29-1b8a-44f7-a2c3-867063f66915 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:03.566437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 694/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:03.626271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596ee2-3e31-4ab3-9141-2194df756226 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:03.626608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596ee2-3e31-4ab3-9141-2194df756226 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:03.626875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596ee2-3e31-4ab3-9141-2194df756226 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:03.628438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 695/1389] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:36:03.634050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01fbca-3362-431f-875f-7fe11f91fee4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:03.634345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01fbca-3362-431f-875f-7fe11f91fee4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:03.634962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01fbca-3362-431f-875f-7fe11f91fee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:03.715524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0a01fbca-3362-431f-875f-7fe11f91fee4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:03.716891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0a01fbca-3362-431f-875f-7fe11f91fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:03.823180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 695/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:04.123114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.123616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.123802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.125261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 696/1391] 10.222.0.60 () {54 vars in 962 bytes} [Wed Sep 28 15:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:04.130954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944b2-0629-4d5f-9392-231552cd3061 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.131246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944b2-0629-4d5f-9392-231552cd3061 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.131527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944b2-0629-4d5f-9392-231552cd3061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.147147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:04.208999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f7944b2-0629-4d5f-9392-231552cd3061 None None] MFA Rules not processed for user `25cbe9fdae7b41ad9735c20260ffe22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:04.210519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f7944b2-0629-4d5f-9392-231552cd3061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:04.287765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 696/1392] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Sep 28 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:04.297455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:04.299007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:04.331530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:04.413607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1552f-3d37-4887-938f-952eae9efdfc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:04.492353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:04.492996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.493274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.493546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.495234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:04.578315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:04.611253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:04.612697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fc21-f45c-42ac-8086-06b33eba79d2 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:04.630838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 697/1393] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:05.385196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882814ef-de8b-4663-88d4-78c02161e5d0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.385738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882814ef-de8b-4663-88d4-78c02161e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.385738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882814ef-de8b-4663-88d4-78c02161e5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.455084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-882814ef-de8b-4663-88d4-78c02161e5d0 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:05.456300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-882814ef-de8b-4663-88d4-78c02161e5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:05.511789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:05.546297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 697/1394] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:05.594442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237b35c-884f-4740-a41b-78513f693f11 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.594922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237b35c-884f-4740-a41b-78513f693f11 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.595017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237b35c-884f-4740-a41b-78513f693f11 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.596385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 698/1395] 10.222.0.60 () {54 vars in 943 bytes} [Wed Sep 28 15:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:05.603088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.603380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.603663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.686880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] MFA Rules not processed for user `8daf1537e6fa46238223fd03ced27a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:05.688134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:05.787356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4792-c949-430e-b8b6-9bc38610db1b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.787630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4792-c949-430e-b8b6-9bc38610db1b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.787910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4792-c949-430e-b8b6-9bc38610db1b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.789240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 699/1396] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:36:05.794469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.794747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.795013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.798681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 698/1397] 10.222.0.60 () {58 vars in 1040 bytes} [Wed Sep 28 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:05.808251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:05.818739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d51d2-2c5a-4e21-af17-68073dea16c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:05.848058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:05.864765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:05.865989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:05.932914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:05.933527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:05.933808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:05.934455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:05.936452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:05.967527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 700/1398] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:06.020232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:06.021818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c9d10-e0ec-4960-abaf-c47b3b0b46d7 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:06.029657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 699/1399] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:36:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:36:06.264162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:06.265436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:06.307514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:06.391582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49840be0-bad6-43fd-a46b-a7d91e983d30 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:06.518999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:06.519607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:06.519886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:06.520156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:06.522997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:06.560301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:06.623546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:06.625154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e0d7f-88ec-4833-b0c5-213bd69a77aa service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:06.644015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 701/1400] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:08.977718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:08.978689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:08.981168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:08.983346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 700/1401] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:08.989530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8a76d-0eb5-4d5d-8835-ea840fd83204 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:08.991117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8a76d-0eb5-4d5d-8835-ea840fd83204 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:08.991117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8a76d-0eb5-4d5d-8835-ea840fd83204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:09.070771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ebd8a76d-0eb5-4d5d-8835-ea840fd83204 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:09.071991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebd8a76d-0eb5-4d5d-8835-ea840fd83204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:09.179985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 702/1402] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:09.516172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:09.517613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:09.587582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:09.694927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32763e-2743-49f7-b54f-b4b55bcabdd3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:09.823213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:09.823827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:09.824105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:09.824373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:09.825997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:09.847285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:09.931152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:09.932842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ac9bf-0800-468a-9566-97c2d88293f3 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:09.953464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 701/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:12.180575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:12.181133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:12.181133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:12.182595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 703/1404] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:12.188223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dbeb8-a3e1-4fff-bb03-f1813459842d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:12.188639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dbeb8-a3e1-4fff-bb03-f1813459842d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:12.188828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dbeb8-a3e1-4fff-bb03-f1813459842d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:12.261433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ea7dbeb8-a3e1-4fff-bb03-f1813459842d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:12.265171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea7dbeb8-a3e1-4fff-bb03-f1813459842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:12.362837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:12.370035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 702/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:12.662950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:12.664467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:12.678399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:12.833026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d293b-3888-4e26-a06a-5aedc1e27f2e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:12.947493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:12.975261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:12.975261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:12.975261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:12.975261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:12.976349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:13.095190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:13.096795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d22f-9f8d-4f45-83c2-1151bf817a23 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:13.116158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 704/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:15.339963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:15.340511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:15.340511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:15.342479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 703/1407] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:15.347716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0250-9f7f-4e1b-b75b-f6b7ce0d0185 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:15.348101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0250-9f7f-4e1b-b75b-f6b7ce0d0185 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:15.348275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0250-9f7f-4e1b-b75b-f6b7ce0d0185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:15.428386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f96e0250-9f7f-4e1b-b75b-f6b7ce0d0185 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:15.429908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f96e0250-9f7f-4e1b-b75b-f6b7ce0d0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:15.485006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:15.540754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 705/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:15.831101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:15.832731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:16.000572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e3b4e-8c48-451b-a99a-492599a9a1c4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:16.072906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:16.125856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:16.126489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:16.126777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:16.127046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:16.128686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:16.230352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:16.232615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980929e-fdb6-4628-96a9-7964fe58e337 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:16.257365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 704/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:25.539446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=23 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:26.255221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=64 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:28.757950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b841ee5-106b-46ce-a56c-aa980e082ae9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.758545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b841ee5-106b-46ce-a56c-aa980e082ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.758545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b841ee5-106b-46ce-a56c-aa980e082ae9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.759880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 706/1410] 10.222.0.60 () {54 vars in 918 bytes} [Wed Sep 28 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:28.765592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54924145-a488-417d-9aeb-82a31538be1a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.766065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54924145-a488-417d-9aeb-82a31538be1a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.766171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54924145-a488-417d-9aeb-82a31538be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.842116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-54924145-a488-417d-9aeb-82a31538be1a None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:28.843347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-54924145-a488-417d-9aeb-82a31538be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:28.949994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 705/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Sep 28 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:28.964347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4155d8-77ff-40c2-a2ce-61099a22d5f0 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.964630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4155d8-77ff-40c2-a2ce-61099a22d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.964907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4155d8-77ff-40c2-a2ce-61099a22d5f0 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.966269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 707/1412] 10.222.0.60 () {54 vars in 918 bytes} [Wed Sep 28 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:28.970871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340cce-8345-441f-b5b7-318f9299111e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.971161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340cce-8345-441f-b5b7-318f9299111e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.971449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340cce-8345-441f-b5b7-318f9299111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:29.044501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:29.047813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f340cce-8345-441f-b5b7-318f9299111e None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:29.049547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f340cce-8345-441f-b5b7-318f9299111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:29.163475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 706/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Sep 28 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:31.773604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:31.774170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:31.774170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:31.775514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:31.780766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecfc-9393-4367-b452-9e1925df0820 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:31.781199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecfc-9393-4367-b452-9e1925df0820 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:31.781314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecfc-9393-4367-b452-9e1925df0820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:31.846756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ce5aecfc-9393-4367-b452-9e1925df0820 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:31.847960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ce5aecfc-9393-4367-b452-9e1925df0820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:31.902350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:31.946304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:32.235297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:32.236935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:32.395554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b1e54-a972-4832-ae5e-49daf73c7a2d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:32.468711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:32.508146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:32.508670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:32.508948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:32.509218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:32.510861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:32.598368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:32.599976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371357-55d8-4152-9c0d-3da38e0af883 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:32.617319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 709/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:33.757610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:33.758213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:33.758213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:33.759516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:33.765795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdec56-d796-488b-b9c7-b0ef6578c365 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:33.766282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdec56-d796-488b-b9c7-b0ef6578c365 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:33.766388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdec56-d796-488b-b9c7-b0ef6578c365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:33.839735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8ecdec56-d796-488b-b9c7-b0ef6578c365 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:33.840933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ecdec56-d796-488b-b9c7-b0ef6578c365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:33.867820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:33.936149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:34.087951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:34.089489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:34.232967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a565f8-1765-4e24-a3a8-c41c7d00db2a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:34.276628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:34.357616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:34.358599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:34.358599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:34.358817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:34.360423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:34.451726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:34.453329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42969e14-4ca9-49f1-8085-64b38eef87ba service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:34.470871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 709/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:35.605300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:35.605815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:35.605815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:35.607184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:35.613977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e23d-6025-49a8-8b21-69ca3d3413ae None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:35.614470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e23d-6025-49a8-8b21-69ca3d3413ae None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:35.614578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e23d-6025-49a8-8b21-69ca3d3413ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:35.667250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:35.690255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bb96e23d-6025-49a8-8b21-69ca3d3413ae None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:35.691487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb96e23d-6025-49a8-8b21-69ca3d3413ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:35.787726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:35.941775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:35.943321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:36.085568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377512a-c3af-4dfe-8086-88fac499aee5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:36.097774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:36.197604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:36.198114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:36.198386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:36.198679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:36.200257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:36.287785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:36.289372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3debce-7721-4a43-bcb4-f8f469832f91 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:36.307856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 712/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:37.424358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78776-c314-40ad-bcc3-808163adacce None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:37.424869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78776-c314-40ad-bcc3-808163adacce None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:37.424965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78776-c314-40ad-bcc3-808163adacce None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:37.426275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:37.432819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1a7e1-e98f-493f-ba64-169b77d89db5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:37.433248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1a7e1-e98f-493f-ba64-169b77d89db5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:37.433390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1a7e1-e98f-493f-ba64-169b77d89db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:37.453391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:37.509095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-aed1a7e1-e98f-493f-ba64-169b77d89db5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:37.510298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aed1a7e1-e98f-493f-ba64-169b77d89db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:37.604882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:37.754937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e78776-c314-40ad-bcc3-808163adacce None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:37.756380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75e78776-c314-40ad-bcc3-808163adacce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:37.871981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:37.908917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e78776-c314-40ad-bcc3-808163adacce None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:38.019842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:38.020446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:38.020724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:38.020991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:38.022608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:38.099940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:38.113363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:38.115075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ff708-b5ef-4a35-bfec-534a50ef3d80 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:38.133277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 712/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:39.256859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08327212-4486-4f49-8080-64e063828a11 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:39.257426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08327212-4486-4f49-8080-64e063828a11 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:39.257426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08327212-4486-4f49-8080-64e063828a11 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:39.258827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:39.266527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186d70b-0c51-49b6-884a-d36bb5de4e96 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:39.267017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186d70b-0c51-49b6-884a-d36bb5de4e96 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:39.267492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186d70b-0c51-49b6-884a-d36bb5de4e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:39.343517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2186d70b-0c51-49b6-884a-d36bb5de4e96 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:39.344713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2186d70b-0c51-49b6-884a-d36bb5de4e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:39.438241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:39.589230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08327212-4486-4f49-8080-64e063828a11 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:39.591622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-08327212-4486-4f49-8080-64e063828a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:39.674500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:39.736477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08327212-4486-4f49-8080-64e063828a11 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:39.848762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:39.849271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:39.849547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:39.849821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:39.851453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:39.903351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:39.955714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:39.957312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288844d-d406-4fef-afa9-3e72df97f592 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:39.975534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 715/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:41.043926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:41.044713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:41.045399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:41.046902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:41.053137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9c72-6778-43e5-b2d6-8fbc0ebd9039 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:41.053864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9c72-6778-43e5-b2d6-8fbc0ebd9039 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:41.053963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9c72-6778-43e5-b2d6-8fbc0ebd9039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:41.128297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0e7e9c72-6778-43e5-b2d6-8fbc0ebd9039 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:41.129511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0e7e9c72-6778-43e5-b2d6-8fbc0ebd9039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:41.222514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:41.379103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:41.380618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:41.429606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:41.527256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796aaf3-e9ff-451f-9c2c-461fceaa79e3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:41.640177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:41.640695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:41.640976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:41.641248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:41.642891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:41.649732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:41.732102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:41.733727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa36173-521f-4fda-b7d4-7c6ad1256812 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:41.751409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 715/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:42.819026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:42.819747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:42.819747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:42.820949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:42.827446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef059-1311-45d3-9279-10b03baa16b2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:42.828023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef059-1311-45d3-9279-10b03baa16b2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:42.828314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef059-1311-45d3-9279-10b03baa16b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:42.905178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fb8ef059-1311-45d3-9279-10b03baa16b2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:42.906394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb8ef059-1311-45d3-9279-10b03baa16b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:42.974796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:43.003588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:43.155363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:43.156882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:43.173796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:43.306533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf661912-f054-4942-abcb-04d003f4ea3a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:43.394920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:43.421713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:43.422241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:43.422550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:43.422824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:43.424432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:43.512814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:43.514983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0fe3-35e9-4d8a-976f-2a3e6598459e service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:43.532444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 718/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:44.576823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:44.577391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:44.577391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:44.578659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:44.584904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4648e-12a4-42a7-a872-dd8ae7aa1487 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:44.585349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4648e-12a4-42a7-a872-dd8ae7aa1487 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:44.585465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4648e-12a4-42a7-a872-dd8ae7aa1487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:44.660249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5fe4648e-12a4-42a7-a872-dd8ae7aa1487 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:44.661465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5fe4648e-12a4-42a7-a872-dd8ae7aa1487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:44.703239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:44.757808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:44.909503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:44.911076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:45.061156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7103d3-f7d8-4f40-bcf1-8538db471e71 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:45.115096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:45.174384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:45.174933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:45.175212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:45.175482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:45.177082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:45.271714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:45.273196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb09128-9e4f-4544-bf90-8b5133bd3eb6 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:45.291192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 718/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:46.357591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:46.358149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:46.358149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:46.359434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:46.365803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d74f-e2c1-48c6-b21e-c442008b9847 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:46.366242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d74f-e2c1-48c6-b21e-c442008b9847 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:46.366339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d74f-e2c1-48c6-b21e-c442008b9847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:46.442376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8ad2d74f-e2c1-48c6-b21e-c442008b9847 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:46.443719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8ad2d74f-e2c1-48c6-b21e-c442008b9847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:46.445700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:46.539330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:46.691429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:46.693040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:46.839823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5e6eb-0a89-4756-b323-487431bb4303 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:46.866370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:46.954169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:46.954719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:46.955004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:46.955272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:46.956878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:47.043375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:47.045453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a59a-3a7a-44b3-9ac2-a0c5f0e3e478 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:47.064114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 721/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:49.112145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:49.113104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:49.113104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:49.114712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:49.119956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d406-1556-4718-8e50-d4ba40d337ad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:49.120455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d406-1556-4718-8e50-d4ba40d337ad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:49.120563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d406-1556-4718-8e50-d4ba40d337ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:49.161666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:49.187513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1b52d406-1556-4718-8e50-d4ba40d337ad None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:49.188718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1b52d406-1556-4718-8e50-d4ba40d337ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:49.284807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:49.584022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:49.585552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:49.693489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72749d9-69b2-4c97-8a8d-6d7ad836f5ae None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:49.725496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:49.768623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:49.769262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:49.769535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:49.769805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:49.771457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:49.879969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:49.881308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e094791-2d6a-4f20-9622-0503a0932016 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:49.902041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 721/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:52.231185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:52.231750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:52.231750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:52.233098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:52.238928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35107-4ba2-4347-82c0-4a8d36256c22 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:52.239367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35107-4ba2-4347-82c0-4a8d36256c22 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:52.239469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35107-4ba2-4347-82c0-4a8d36256c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:52.314662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-43d35107-4ba2-4347-82c0-4a8d36256c22 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:52.315775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-43d35107-4ba2-4347-82c0-4a8d36256c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:52.329336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:52.418024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:52.704226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:52.705792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:52.809159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758783-ebc3-4f39-84bf-1f94f9e22a15 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:52.822431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:52.884486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:52.885090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:52.885367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:52.885635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:52.887291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:52.995087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:52.996437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750866fa-9365-4234-b691-efd11bd55c87 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:53.016156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 724/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:55.589142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:55.589701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:55.589701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:55.590971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:55.596252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dc211-2b06-4a0d-8398-2e3e0c8d2a45 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:55.596679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dc211-2b06-4a0d-8398-2e3e0c8d2a45 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:55.596831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dc211-2b06-4a0d-8398-2e3e0c8d2a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:55.644332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:55.661717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7d6dc211-2b06-4a0d-8398-2e3e0c8d2a45 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:55.662951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7d6dc211-2b06-4a0d-8398-2e3e0c8d2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:55.757327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:56.049899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:56.051497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:56.161138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59b0e3-3e6e-475d-a2be-566edb103dd5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:56.203750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:56.238632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:56.239243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:56.239525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:56.239803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:56.241425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:56.351310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:56.352684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b9588-6037-42dc-beb4-00c590f5b562 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:56.371057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 724/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:58.657679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:58.658255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:58.658255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:58.659622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:58.665184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716dc6-8f2c-49bb-8e7d-7720d5fec7fb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:58.665619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716dc6-8f2c-49bb-8e7d-7720d5fec7fb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:58.665731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716dc6-8f2c-49bb-8e7d-7720d5fec7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:58.738730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e6716dc6-8f2c-49bb-8e7d-7720d5fec7fb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:58.739956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e6716dc6-8f2c-49bb-8e7d-7720d5fec7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:58.762035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:58.844498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:59.134522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:59.136064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:59.249493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab20e-be01-4c23-a7a1-dc1a779bb15c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:59.272743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:59.328971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:59.329566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:59.329842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:59.330112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:59.331751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:59.441686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:59.443149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb990b2c-04a5-43cd-a005-651a9fdf3719 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:59.460781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 727/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:01.766335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:01.766940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:01.766940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:01.768328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:01.773593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ea72-0738-43bf-a835-8cd49f0925ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:01.773996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ea72-0738-43bf-a835-8cd49f0925ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:01.774126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ea72-0738-43bf-a835-8cd49f0925ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:01.838193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:01.840769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cf48ea72-0738-43bf-a835-8cd49f0925ee None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:01.842662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cf48ea72-0738-43bf-a835-8cd49f0925ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:01.960345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:02.248425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:02.249970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:02.354150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea54aca-86f0-47cb-8ecc-4d725eeb8c26 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:02.402869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:02.429342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:02.429955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:02.430235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:02.430543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:02.432148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:02.539641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:02.541088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916538c-8f88-492b-920d-a47111b53166 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:02.559838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 727/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:04.837123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:04.837688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:04.837688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:04.839033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:04.844943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026d6e-1637-4fba-aca9-68045be86061 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:04.845441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026d6e-1637-4fba-aca9-68045be86061 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:04.845548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026d6e-1637-4fba-aca9-68045be86061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:04.922310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ac026d6e-1637-4fba-aca9-68045be86061 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:04.924051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ac026d6e-1637-4fba-aca9-68045be86061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:04.958454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:05.025268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:05.311758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:05.313314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:05.417961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26404af-f688-4060-8519-6c76c4b6255f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:05.446560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:05.492023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:05.492649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:05.492931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:05.493206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:05.494858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:05.603274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:05.604622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd533d-517d-4682-8b4a-ea821c3c90cc service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:05.622759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 730/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:07.834624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:07.835201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:07.835201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:07.836498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:07.842496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4817-b5e2-4cb8-ae0a-7e9c5b115d93 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:07.842891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4817-b5e2-4cb8-ae0a-7e9c5b115d93 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:07.843055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4817-b5e2-4cb8-ae0a-7e9c5b115d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:07.910051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-db5a4817-b5e2-4cb8-ae0a-7e9c5b115d93 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:07.911313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-db5a4817-b5e2-4cb8-ae0a-7e9c5b115d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:07.919919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:08.008535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:08.292989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:08.294566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:08.398985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11918ba-7087-4862-bf94-61760b0b87ea None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:08.455622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:08.473620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:08.474222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:08.474533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:08.474813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:08.476403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:08.587342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:08.588704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93f86c-49e9-4f60-b893-5e9d55a7122f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:08.607014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 730/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:10.809582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:10.810165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:10.810165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:10.811538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:10.816989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c2d0-885e-4805-97db-5885a34772fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:10.817445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c2d0-885e-4805-97db-5885a34772fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:10.817567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c2d0-885e-4805-97db-5885a34772fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:10.883176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-79b1c2d0-885e-4805-97db-5885a34772fc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:10.884447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79b1c2d0-885e-4805-97db-5885a34772fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:10.930314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:10.990876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:37:11.281489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:11.283673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:11.386229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffb4fa-f931-4257-b341-3723b64703f1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:11.423098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:11.459804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:11.460411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:11.460687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:11.460956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:11.462600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:11.579913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:11.581241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d107303-9c0a-413c-8ffe-17c72f7d8284 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:11.610614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 733/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:13.767721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:13.768264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:13.768385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:13.769716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:13.775244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee87cd-40b6-4e57-8036-054cbdbe6465 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:13.775666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee87cd-40b6-4e57-8036-054cbdbe6465 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:13.775798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee87cd-40b6-4e57-8036-054cbdbe6465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:13.840984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-18ee87cd-40b6-4e57-8036-054cbdbe6465 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:13.842820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18ee87cd-40b6-4e57-8036-054cbdbe6465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:13.860311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:13.940006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:14.231210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:14.232778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:14.343395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0bcbe-ccce-414b-bbaf-2c4d6d36af40 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:14.411055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:14.418064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:14.418705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:14.418981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:14.419254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:14.420928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:14.531705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:14.533140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c56957-87d7-4369-9191-a1af5c313ece service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:14.551137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 733/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:16.770108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:16.770682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:16.770682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:16.772051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:16.777397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c3fc-2df2-41fa-92a5-059e180cb6e2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:16.777845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c3fc-2df2-41fa-92a5-059e180cb6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:16.777953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c3fc-2df2-41fa-92a5-059e180cb6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:16.847536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bb41c3fc-2df2-41fa-92a5-059e180cb6e2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:16.848772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb41c3fc-2df2-41fa-92a5-059e180cb6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:16.896803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:16.945030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:17.230698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:17.232238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:17.339036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877df03f-3cab-4cce-9240-79edfbfccbb7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:17.386372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:17.412068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:17.412663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:17.412936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:17.413204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:17.414830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:17.533707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:17.536022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497424b6-04c7-48a2-8b90-fb33347b8065 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:17.555224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 736/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:26.943524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=22 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:27.552826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=66 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:37.660782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f23a1-ceb0-4898-8863-5a3d43960b89 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:37.661348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f23a1-ceb0-4898-8863-5a3d43960b89 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:37.661348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f23a1-ceb0-4898-8863-5a3d43960b89 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:37.662664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 735/1471] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:37.667836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60950085-4465-472f-8713-f0a0b35d1e58 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:37.668259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60950085-4465-472f-8713-f0a0b35d1e58 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:37.668415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60950085-4465-472f-8713-f0a0b35d1e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:37.736204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-60950085-4465-472f-8713-f0a0b35d1e58 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:37.737426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60950085-4465-472f-8713-f0a0b35d1e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:37.831793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 737/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:38.132352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:38.132636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:38.132911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:38.134228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 736/1473] 10.222.0.60 () {54 vars in 962 bytes} [Wed Sep 28 15:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:38.139266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311443-5262-4a34-8b69-9338b6426bba None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:38.139556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311443-5262-4a34-8b69-9338b6426bba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:38.139833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311443-5262-4a34-8b69-9338b6426bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:38.218741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0b311443-5262-4a34-8b69-9338b6426bba None None] MFA Rules not processed for user `25cbe9fdae7b41ad9735c20260ffe22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:38.219937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0b311443-5262-4a34-8b69-9338b6426bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:38.240457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:38.298283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 738/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Sep 28 15:37:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:38.306065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:38.307726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:38.410520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7ab1b-3f50-46f2-ba57-5b8a5b92be46 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:38.493229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:38.493836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:38.494117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:38.494389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:38.496024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:38.551301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:38.607924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:38.609359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b5a12-7e00-473e-a1a8-1fc7f1884924 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:38.628592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 737/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:40.881987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:40.882575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:40.882575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:40.883934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 739/1476] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:40.889784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960b598-f548-4805-aa3a-e71428b2f81d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:40.890287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960b598-f548-4805-aa3a-e71428b2f81d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:40.890385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960b598-f548-4805-aa3a-e71428b2f81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:40.956544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8960b598-f548-4805-aa3a-e71428b2f81d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:40.957788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8960b598-f548-4805-aa3a-e71428b2f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:41.052067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 738/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:41.333019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:41.334596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:41.465910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905e2f2-f146-42ba-ba51-ff0580d6d429 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:41.517899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:41.562243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:41.562916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:41.563188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:41.563451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:41.564081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:41.565812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:37:41.565812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:37:41.565812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:37:41.566966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c1f6d-7aad-4e37-be67-b0e57577413c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:41.586052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 740/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 15:37:41 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:37:43.787883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:43.788514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:43.788514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:43.789774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 739/1479] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:43.795915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fb1-dbd4-4619-8091-c24fbfb76836 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:43.796339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fb1-dbd4-4619-8091-c24fbfb76836 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:43.796450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fb1-dbd4-4619-8091-c24fbfb76836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:43.862661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-442d8fb1-dbd4-4619-8091-c24fbfb76836 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:43.863877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-442d8fb1-dbd4-4619-8091-c24fbfb76836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:43.958669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 741/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:37:44.251347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:44.252850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:44.262396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:44.399572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9554f-79f7-44a9-843d-245403ec904f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:44.478975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:44.510859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:44.511463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:44.511734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:44.512001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:44.514191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:44.601495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:44.603129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755ca03-cd9f-4334-862e-a6824b34635c service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:44.620715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 740/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:46.942344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:46.942954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:46.942954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:46.944259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 742/1482] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:46.949666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799999c-cc14-471c-b67c-c9dad968f312 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:46.950097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799999c-cc14-471c-b67c-c9dad968f312 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:46.950256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799999c-cc14-471c-b67c-c9dad968f312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:47.017457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2799999c-cc14-471c-b67c-c9dad968f312 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:47.018696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2799999c-cc14-471c-b67c-c9dad968f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:47.070897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:47.121586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 741/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:37:47.410832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:47.412390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:47.436696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:47.525457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68312fff-facb-4cc9-a11a-5738ec4e6cb8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:47.601023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:47.601660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:47.601936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:47.602208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:47.603851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:47.680817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:47.715726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:47.717534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae92d2c-f7a8-4f76-a03b-b2381e8ee5d3 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:47.735744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 743/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:50.037947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:50.038497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:50.038497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:50.039774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 742/1485] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:50.045038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afaf55-26dd-4d67-8b77-04602a2d5705 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:50.045468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afaf55-26dd-4d67-8b77-04602a2d5705 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:50.045587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afaf55-26dd-4d67-8b77-04602a2d5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:50.111140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-73afaf55-26dd-4d67-8b77-04602a2d5705 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:50.112613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73afaf55-26dd-4d67-8b77-04602a2d5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:50.207516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 744/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:37:50.509056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:50.510665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:50.613039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde69025-3b42-4524-930f-8f5477676fbb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:50.683785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:50.685837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:50.686536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:50.687008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:50.687478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:50.689305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:50.801008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:50.802471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4626-3eab-4f2b-9d19-eb1d55f2a97c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:50.821245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 743/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:53.155223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:53.155813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:53.155813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:53.157108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 745/1488] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:53.162384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4540b0a-e760-45b7-aa14-3008249aca86 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:53.162860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4540b0a-e760-45b7-aa14-3008249aca86 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:53.162967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4540b0a-e760-45b7-aa14-3008249aca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:53.231254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f4540b0a-e760-45b7-aa14-3008249aca86 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:53.232464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f4540b0a-e760-45b7-aa14-3008249aca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:53.287293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:53.332461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 744/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:37:53.624320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:53.625934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:53.661155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:53.736637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4c7ed-dfd2-4592-a635-e7ecf9a061e7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:53.809237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:53.809850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:53.810135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:53.811100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:53.812753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:53.891772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:53.922401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:53.924078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6e4ce-7dc1-431d-923e-a295f6c9865f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:53.944235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 746/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:37:56.291911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:56.292421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:56.292532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:56.293787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 745/1491] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 15:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:37:56.300414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854a62-b464-4663-8874-98f2a85b0dde None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:56.300936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854a62-b464-4663-8874-98f2a85b0dde None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:56.301042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854a62-b464-4663-8874-98f2a85b0dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:56.367182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-61854a62-b464-4663-8874-98f2a85b0dde None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:37:56.368294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-61854a62-b464-4663-8874-98f2a85b0dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:56.463245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 747/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:37:56.760983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:37:56.762545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:56.869634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629542-90c6-46b8-b06a-afdcea13a5b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:37:56.952721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:37:56.953326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:37:56.953607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:37:56.953882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:37:56.955529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:37:56.964512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:37:57.066107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:37:57.067499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c67531-c3f2-4b42-a09b-d825cf390b2e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:37:57.086516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 746/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:38:06.461677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=82 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:07.084161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:10.361052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291068-7351-43b6-a2b6-105d8013024e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:10.361608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291068-7351-43b6-a2b6-105d8013024e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:10.361608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291068-7351-43b6-a2b6-105d8013024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:10.438499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-38291068-7351-43b6-a2b6-105d8013024e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:10.440008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-38291068-7351-43b6-a2b6-105d8013024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:10.539787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 748/1494] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:10.547776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:10.548290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:10.548400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:10.626124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:10.627298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:10.724233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 747/1495] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:10.728915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291068-7351-43b6-a2b6-105d8013024e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:10.730129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-38291068-7351-43b6-a2b6-105d8013024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:10.741465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291068-7351-43b6-a2b6-105d8013024e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:10.806442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:10.841051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:10.841648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:10.841952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:10.842217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:10.842884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:39:10.843186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:10.844190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:10.844190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:39:10.844190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:39:10.845062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:10.857161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 749/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:39:10.865577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:10.875612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848efd-fc2e-4984-9e95-92a8df6ce417 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:10.940016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:10.973067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:10.973673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:10.973952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:10.974223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:10.974890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:10.976654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:10.976654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:39:10.976654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:39:10.977858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:11.032859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 748/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:39:10 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:39:11.037040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:11.038592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:11.072208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93ef2-2538-40cf-9110-a4e0935cba75 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:11.164539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:11.165143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.165420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.165694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.166344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:11.167423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:11.167423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:39:11.167423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:39:11.169305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:11.218122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 750/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:39:11 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:39:11.222480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:11.224082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:11.288479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5f9d-e599-4cae-ba6d-51d0c0592690 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:11.371991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:11.383649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:11.384263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.384546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.384821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.385499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:11.386533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:11.386533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:39:11.386533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:39:11.387422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:11.399557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 749/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:39:11.404080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:11.405596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:11.416903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96859b4f-8998-457b-8dcd-5cfd96371dce admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:11.442044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:11.512132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:11.512647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.512925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.513195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.513839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:11.514889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:11.514889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:39:11.514889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:39:11.515766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277cdfb-d564-4420-9da6-c7edb1425637 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:11.528442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 751/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:39:11.532680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:11.534208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:11.568985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1352f5-7726-4f58-befd-a745c835d4c6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:11.780943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:11.781522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.781805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.782076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] PATH_INFO: `/v3/projects/95ea6abc5bdf44f1b2988f3da45fac40/users/415e5d2235364558a23b6cc86adab2f8/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.805712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ea6abc5bdf44f1b2988f3da45fac40, user_id=415e5d2235364558a23b6cc86adab2f8, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:11.806783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:11.806783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:39:11.806783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:39:11.808994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba08e6-05fb-468c-bc7a-3f5f88859bb6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:11.833719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 750/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:39:11 2022] PUT /identity/v3/projects/95ea6abc5bdf44f1b2988f3da45fac40/users/415e5d2235364558a23b6cc86adab2f8/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:39:11.846432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce682-cbce-45da-aa40-b8a5ee7b28fe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.846726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce682-cbce-45da-aa40-b8a5ee7b28fe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.847002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce682-cbce-45da-aa40-b8a5ee7b28fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.904670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a2bce682-cbce-45da-aa40-b8a5ee7b28fe None None] MFA Rules not processed for user `415e5d2235364558a23b6cc86adab2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:11.905921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a2bce682-cbce-45da-aa40-b8a5ee7b28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:11.982282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 752/1502] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:39:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:11.995582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:11.995871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:11.996152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:11.997993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 751/1503] 10.222.0.60 () {54 vars in 963 bytes} [Wed Sep 28 15:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:12.002897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b4a0-33be-4a5e-bea6-03c286dd45b0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:12.003181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b4a0-33be-4a5e-bea6-03c286dd45b0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:12.003445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b4a0-33be-4a5e-bea6-03c286dd45b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:12.034963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:12.078911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0ba2b4a0-33be-4a5e-bea6-03c286dd45b0 None None] MFA Rules not processed for user `9685e7e865604d4b9acd36902045d141`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:12.080101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0ba2b4a0-33be-4a5e-bea6-03c286dd45b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:12.155339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 753/1504] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Sep 28 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:12.164476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:12.165994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:12.216132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:12.270615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2e6ce-b73e-467c-a47d-053517685c18 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:12.343238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:12.343765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:12.344045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:12.344321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:12.345993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:12.442471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:12.443906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:12.447278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:12.461923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 752/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:12.507997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f35a8-7996-40f6-8649-6a0a1da32301 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:12.508449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f35a8-7996-40f6-8649-6a0a1da32301 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:12.508568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f35a8-7996-40f6-8649-6a0a1da32301 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:12.509811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 754/1506] 10.222.0.60 () {52 vars in 830 bytes} [Wed Sep 28 15:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:12.513612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:12.514864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:12.613333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d448989-8109-469a-b5b1-bf0d2bade97b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:12.707558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:12.708176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:12.708451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:12.708719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] PATH_INFO: `/v3/users/415e5d2235364558a23b6cc86adab2f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:12.721682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415e5d2235364558a23b6cc86adab2f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:12.722724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:12.722724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:39:12.722724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:39:12.723921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:12.733850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:12.788256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415e5d2235364558a23b6cc86adab2f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:12.804492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415e5d2235364558a23b6cc86adab2f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:12.804979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415e5d2235364558a23b6cc86adab2f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:12.805505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415e5d2235364558a23b6cc86adab2f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:12.814991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415e5d2235364558a23b6cc86adab2f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:12.844609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 753/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:39:12 2022] DELETE /identity/v3/users/415e5d2235364558a23b6cc86adab2f8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:39:12.849132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:12.849411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:12.849681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:12.907460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:12.908691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:12.983632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:12.997202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 755/1508] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:13.006833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:13.008051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:13.105993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e622e7-3b59-4f05-8242-f6ebbacfd5a0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:13.162575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:13.181659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:13.182297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:13.182855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:13.183129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:13.184743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:13.293743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:13.295222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694a239-190b-4ead-9253-a1f7f5f9b7ce service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:13.313103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 754/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:13.641001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:13.642322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:13.654035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56663ad9-8df3-4644-bf01-ff482b7e15db None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:13.750289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:13.750857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:13.751132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:13.751401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] PATH_INFO: `/v3/projects/95ea6abc5bdf44f1b2988f3da45fac40` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:13.761604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ea6abc5bdf44f1b2988f3da45fac40)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:13.763286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:39:13.763286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:39:13.763286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:39:13.764478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:13.824398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ea6abc5bdf44f1b2988f3da45fac40', 'request_id': 'req-c806460a-47d6-4f07-a443-ed7adc795515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:13.835219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ea6abc5bdf44f1b2988f3da45fac40', 'request_id': 'req-c806460a-47d6-4f07-a443-ed7adc795515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:13.848590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ea6abc5bdf44f1b2988f3da45fac40', 'request_id': 'req-c806460a-47d6-4f07-a443-ed7adc795515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:13.871224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] The token cache is being invalidate because project 95ea6abc5bdf44f1b2988f3da45fac40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:39:13.872089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:13.872526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c806460a-47d6-4f07-a443-ed7adc795515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:39:13.873920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 756/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:39:13 2022] DELETE /identity/v3/projects/95ea6abc5bdf44f1b2988f3da45fac40 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:39:23.310809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=62 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:23.869067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:40:44.234755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:44.235342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:44.235342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:44.236702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:40:44.241509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d80a-aee7-4671-a481-760bf4d5e772 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:44.241963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d80a-aee7-4671-a481-760bf4d5e772 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:44.242068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d80a-aee7-4671-a481-760bf4d5e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:44.325662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7755d80a-aee7-4671-a481-760bf4d5e772 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:44.327317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7755d80a-aee7-4671-a481-760bf4d5e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:44.432580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:44.590262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:40:44.592368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:44.710869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903a4e2-5b6f-4a58-bc4c-273180c605f1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:40:44.716914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:40:44.717568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:44.717850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:44.718125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:44.726223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:40:44.727848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7d805-d708-4f75-b670-1b0cfbcd2a1e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:40:44.737154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 756/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:40:45.381526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa6a7-8adf-480a-8ce9-9942bcf24a67 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:45.382099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa6a7-8adf-480a-8ce9-9942bcf24a67 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:45.382099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa6a7-8adf-480a-8ce9-9942bcf24a67 None None] PATH_INFO: `/v3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:45.383518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 758/1514] 10.222.0.75 () {54 vars in 925 bytes} [Wed Sep 28 15:40:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:40:45.390468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867ab1-b5e7-46b1-b4d2-6d6d9fac9ead None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:45.390926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867ab1-b5e7-46b1-b4d2-6d6d9fac9ead None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:45.391034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867ab1-b5e7-46b1-b4d2-6d6d9fac9ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:45.453167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5e867ab1-b5e7-46b1-b4d2-6d6d9fac9ead None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:45.454400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5e867ab1-b5e7-46b1-b4d2-6d6d9fac9ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:45.532605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:40:45.538058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 757/1515] 10.222.0.75 () {58 vars in 1015 bytes} [Wed Sep 28 15:40:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:45.847147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ae06-a543-4ed7-95c4-86030d2f49cc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:45.847355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ae06-a543-4ed7-95c4-86030d2f49cc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:45.847638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ae06-a543-4ed7-95c4-86030d2f49cc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:45.848982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 759/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:40:45.855227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e886f-cb59-4b0b-a3aa-8e42a5ddad9e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:45.855606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e886f-cb59-4b0b-a3aa-8e42a5ddad9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:45.855881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e886f-cb59-4b0b-a3aa-8e42a5ddad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:45.875595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a72e886f-cb59-4b0b-a3aa-8e42a5ddad9e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:45.876774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a72e886f-cb59-4b0b-a3aa-8e42a5ddad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:45.885446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 758/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:46.036644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274ae06-a543-4ed7-95c4-86030d2f49cc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:40:46.050765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274ae06-a543-4ed7-95c4-86030d2f49cc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:40:46.056132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:40:46.056721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:46.056999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:46.057270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:46.065558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:40:46.067174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1124a-ace3-461b-bf42-41bf2b16f17e service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:40:46.076815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 760/1518] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:40:54.240877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c925857-5c51-4058-a490-e7caaf16a60f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:54.241336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c925857-5c51-4058-a490-e7caaf16a60f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:54.241427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c925857-5c51-4058-a490-e7caaf16a60f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:54.242788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 759/1519] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:40:54.248132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a701d-8955-4481-95b6-12837033bc49 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:54.249225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a701d-8955-4481-95b6-12837033bc49 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:54.249318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a701d-8955-4481-95b6-12837033bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:54.270113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3a9a701d-8955-4481-95b6-12837033bc49 None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:54.271352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3a9a701d-8955-4481-95b6-12837033bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:54.279833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 761/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:40:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:54.287489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c925857-5c51-4058-a490-e7caaf16a60f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:40:54.301326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c925857-5c51-4058-a490-e7caaf16a60f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:40:54.306154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:40:54.306784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:54.307061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:54.307327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:54.316375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:40:54.317981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a17c7-b202-4078-98f7-0f4fbb527b34 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:40:54.327893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 760/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:40:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:40:57.548447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdfec7-4d60-40e6-8cec-ac65fe6da27e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:57.548969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdfec7-4d60-40e6-8cec-ac65fe6da27e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:57.548969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdfec7-4d60-40e6-8cec-ac65fe6da27e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:57.550349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:40:57.556837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda074fb-5fda-4ba3-964a-42e5bcd467c2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:57.557326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda074fb-5fda-4ba3-964a-42e5bcd467c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:57.557431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda074fb-5fda-4ba3-964a-42e5bcd467c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:57.577646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bda074fb-5fda-4ba3-964a-42e5bcd467c2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:57.578884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bda074fb-5fda-4ba3-964a-42e5bcd467c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:57.586757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:57.744070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdfec7-4d60-40e6-8cec-ac65fe6da27e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:40:57.760036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdfec7-4d60-40e6-8cec-ac65fe6da27e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:40:57.765969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:40:57.766594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:57.766878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:57.767153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:57.775593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:40:57.777195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734160f6-884c-46c4-9583-e58963ea26b5 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:40:57.787566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 763/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:40:58.885884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405073a0-917d-4378-b8f9-dda5d3167a10 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:58.886496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405073a0-917d-4378-b8f9-dda5d3167a10 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:58.886701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405073a0-917d-4378-b8f9-dda5d3167a10 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:58.889386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:40:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:40:58.895934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9cfb-dc57-4c18-b907-c3232990426b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:58.896408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9cfb-dc57-4c18-b907-c3232990426b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:58.896519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9cfb-dc57-4c18-b907-c3232990426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:58.919060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-17ea9cfb-dc57-4c18-b907-c3232990426b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:40:58.920286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17ea9cfb-dc57-4c18-b907-c3232990426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:40:58.928321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:40:59.083894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405073a0-917d-4378-b8f9-dda5d3167a10 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:40:59.098246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405073a0-917d-4378-b8f9-dda5d3167a10 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:40:59.103746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:40:59.104354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:40:59.104637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:40:59.104910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:40:59.113927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:40:59.115547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e70c8-ae6d-4e94-90e0-cc6cb0665ea0 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:40:59.125521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 763/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:03.145979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:03.146569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:03.146569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:03.147942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:03.153368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d19a67-be34-4545-a053-f41673c16314 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:03.153791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d19a67-be34-4545-a053-f41673c16314 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:03.153989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d19a67-be34-4545-a053-f41673c16314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:03.174254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-51d19a67-be34-4545-a053-f41673c16314 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:03.175480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-51d19a67-be34-4545-a053-f41673c16314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:03.183164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:03.479732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:03.481436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:03.516624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:03.563552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733ed9a-947f-4b69-8361-46e6ea528469 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:03.567167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:03.567765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:03.568049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:03.568322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:03.577882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:03.580298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318a647-a1a1-4922-a786-40a7f17e5ca7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:03.590540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 766/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:05.756722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13178-e297-4717-8117-bcc1343654b1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:05.757353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13178-e297-4717-8117-bcc1343654b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:05.757353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13178-e297-4717-8117-bcc1343654b1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:05.758637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:05.763884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10906-d400-4551-a19d-082048529962 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:05.764295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10906-d400-4551-a19d-082048529962 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:05.764433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10906-d400-4551-a19d-082048529962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:05.784155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b3b10906-d400-4551-a19d-082048529962 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:05.785343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b3b10906-d400-4551-a19d-082048529962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:05.793130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:06.084471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13178-e297-4717-8117-bcc1343654b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:06.095640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13178-e297-4717-8117-bcc1343654b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:06.099104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:06.099690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:06.099969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:06.100236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:06.109216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:06.111280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c6764-2af6-499f-8286-feffcb09aa40 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:06.120598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 766/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:09.900896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:09.901467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:09.901467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:09.902885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:09.909745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90a6c-d2b0-468e-8f14-18df254fee64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:09.910220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90a6c-d2b0-468e-8f14-18df254fee64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:09.910313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90a6c-d2b0-468e-8f14-18df254fee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:09.930476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0db90a6c-d2b0-468e-8f14-18df254fee64 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:09.931644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0db90a6c-d2b0-468e-8f14-18df254fee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:09.940932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:10.272457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:10.274131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:10.357974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0284f-c04c-4c0c-93ed-67e67d4775bf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:10.362185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:10.362827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:10.363107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:10.363407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:10.372233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:10.373651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836bbc8-dc27-4fcd-8f92-e70a274591ef service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:10.383251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 769/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:12.534932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:12.535489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:12.535489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:12.536765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:12.541958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8c1f-5e20-4190-863c-e39e0e95c351 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:12.542389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8c1f-5e20-4190-863c-e39e0e95c351 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:12.542544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8c1f-5e20-4190-863c-e39e0e95c351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:12.562237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-980c8c1f-5e20-4190-863c-e39e0e95c351 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:12.563474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-980c8c1f-5e20-4190-863c-e39e0e95c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:12.571181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:12.872075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:12.873770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:12.887259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c8ca6-e8fa-4d37-b2e1-310bc1a5d58c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:12.891297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:12.891899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:12.892184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:12.892477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:12.901617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:12.903069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f234be-2ac0-4e4e-a491-cd4ab4284b3a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:12.912297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 769/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:22.557315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=63 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:22.908195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=37 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:38.996440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366bc3-2261-448c-9bf5-99c2234c96bc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.997087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366bc3-2261-448c-9bf5-99c2234c96bc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.997087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366bc3-2261-448c-9bf5-99c2234c96bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.023292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-03366bc3-2261-448c-9bf5-99c2234c96bc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.024661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-03366bc3-2261-448c-9bf5-99c2234c96bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.032449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc32a-081b-4ecb-bca0-874764627d94 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.033432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc32a-081b-4ecb-bca0-874764627d94 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.033930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc32a-081b-4ecb-bca0-874764627d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.036458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 771/1540] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.058225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.058850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.059052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.064878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-720dc32a-081b-4ecb-bca0-874764627d94 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.066707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-720dc32a-081b-4ecb-bca0-874764627d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.081048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 770/1541] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.089057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.089567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.090087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.090217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.090740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.099474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 772/1542] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:39.105555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.132564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7356a-0686-43af-9f26-59b8d2d56a51 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.134057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.135547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.137809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.138450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.143054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.143381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.144150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.144463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.146221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b8e37-253b-488b-bc99-343c0ab0ad46 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.154063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 771/1543] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:39.159661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.170556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 773/1544] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:42:39.179164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.179464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.179723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.180253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe3ee2-ff19-48e6-b094-676be5c57f6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.203260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.205207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.205729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.206221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.207141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.207829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.209380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:39.209380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:42:39.209380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:42:39.210736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f1c4-db5f-4856-9570-235fcd1573c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.212099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.213319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.230737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 774/1545] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.231469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.242441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 772/1546] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.244848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.245443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.246619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.258763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d3779-0772-4028-81a7-520a5fef3d44 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.269295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.269926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.270204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.270502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.271157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.272183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:39.272183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:42:39.272183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:42:39.273383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0754396-4341-4c68-9dac-2203a8718a96 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.282986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.284377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.292767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 773/1547] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.293368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.304189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f80bd-43f7-43c4-9000-c7cc2de31391 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.316585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.317212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.317490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.318066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.318744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.320660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be737dbb-4fd2-4f2d-b689-e78b0a13cd59 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.325398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 775/1548] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.325398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b7d0d-c2d4-43de-be3a-621a6a02f519 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.325398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b7d0d-c2d4-43de-be3a-621a6a02f519 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.325398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b7d0d-c2d4-43de-be3a-621a6a02f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.345080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b9b7d0d-c2d4-43de-be3a-621a6a02f519 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.353072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b9b7d0d-c2d4-43de-be3a-621a6a02f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.358058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 774/1549] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.359289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.359578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.359852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.381952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.383293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.419014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 776/1550] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.430274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1858a-eaae-414f-8964-6fd4304c61f5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.430274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1858a-eaae-414f-8964-6fd4304c61f5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.430274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1858a-eaae-414f-8964-6fd4304c61f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.450057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27e1858a-eaae-414f-8964-6fd4304c61f5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.451283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27e1858a-eaae-414f-8964-6fd4304c61f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.453733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 775/1551] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.454330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.460759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 777/1552] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.462243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.462557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.462829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.470757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538d0a-0c72-4b34-b4ef-ddf34f4b3fd9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.475414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.476025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.476310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.476588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.477233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.477536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.479201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.482102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.483907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.491986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 776/1553] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.492179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 778/1554] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.499863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.500051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.503522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1617af-4851-482e-b591-bff525dacdd3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.508157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.508754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.509028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.509295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.509891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99760f4-8a80-4633-9e21-8a5f46130135 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.510013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.511877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958ee5-635e-41bd-a10d-0f07cb035ee4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.515210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.515816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.516093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.516371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.516995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.517989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:39.517989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:42:39.517989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:42:39.519238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909c7a9-3cd7-45a8-96d3-80da082aaafa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.533622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 779/1555] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.534924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811cbd-0ee7-4d79-a3ef-afbebc4602f2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.535202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811cbd-0ee7-4d79-a3ef-afbebc4602f2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.535466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811cbd-0ee7-4d79-a3ef-afbebc4602f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.546843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 777/1556] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.548278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad34cf-9982-4926-a551-db97e623c52b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.548565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad34cf-9982-4926-a551-db97e623c52b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.548837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad34cf-9982-4926-a551-db97e623c52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.556554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-25811cbd-0ee7-4d79-a3ef-afbebc4602f2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.557819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-25811cbd-0ee7-4d79-a3ef-afbebc4602f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.566735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 780/1557] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.568130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.568352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.568648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.569568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3cad34cf-9982-4926-a551-db97e623c52b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.570966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3cad34cf-9982-4926-a551-db97e623c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.579867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 778/1558] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.581070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d37dc-c77a-4295-9c32-f369b49aa3a5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.581352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d37dc-c77a-4295-9c32-f369b49aa3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.581633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d37dc-c77a-4295-9c32-f369b49aa3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.588627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.590339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.598453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 781/1559] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.599167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.601904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e51d37dc-c77a-4295-9c32-f369b49aa3a5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.603158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e51d37dc-c77a-4295-9c32-f369b49aa3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.608973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6de225-e0f2-43c7-a8d4-e88580e19732 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.611742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 779/1560] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.613752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.614348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.614661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.614936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.615621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.616236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.616523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.616803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.617524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.636525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.637725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.645886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 780/1561] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.649465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 782/1562] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.649599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.649707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.658643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c40cee-5eda-4479-8e10-274cafbca452 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.685332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:39.710473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:39.716269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0aad5-716e-4f7e-84bb-015b1e25c67a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.717936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.718750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.719286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.719742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.720381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.721709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.722004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.722341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.722665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.722939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.723817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.725532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742b814-82b0-4684-91d7-9e12a52ba68f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.732954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 781/1563] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:39.734284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.736774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 783/1564] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:39.738126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8981-2674-45f2-94da-c020112a1860 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.738437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8981-2674-45f2-94da-c020112a1860 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.738714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8981-2674-45f2-94da-c020112a1860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.751999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29a51-a297-490d-950a-5bc84f019c16 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.757516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.758144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.758435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.758721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.759368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.759670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.761055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-606a8981-2674-45f2-94da-c020112a1860 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.761292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.762293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-606a8981-2674-45f2-94da-c020112a1860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.770208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 784/1565] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.771028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a8981-2674-45f2-94da-c020112a1860 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.774485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 782/1566] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.775739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.780639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a8981-2674-45f2-94da-c020112a1860 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.785463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.785823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570a63c-33a0-459a-8b77-97f2b5bd985e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.786058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.786338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.786647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.787508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.787836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.789429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.790428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.791032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.791310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.791589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.792225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.792648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.794231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.801523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 785/1567] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.802312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.814224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 783/1568] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.814224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.815449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838913d-5a5a-4372-9c34-0727f56c0e4c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.819503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0ef7-8ccf-4a39-b6d9-aed14ff0855d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.820285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.820893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.821169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.821446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.822081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.824064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.824193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.824730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.825008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.825286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.825932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.826233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.828386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.841450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 784/1569] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.842393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.850447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 786/1570] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.850447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.853432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47080-700c-4695-95bc-e08cccfd2502 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.859171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.859772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.860046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.860357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.860466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703cce9-6b95-4088-91b1-78b432b22401 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.861002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.862672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.865460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.866225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.866682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.866973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.867658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.869227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.873630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 785/1571] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:39.875176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.880690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 787/1572] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:39.881425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.892241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83110805-f39a-4354-a850-6908e7f28ae7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.896824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.897424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.897704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.897893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687f85-6e65-4433-8667-d5970e4130af admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.898030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.898769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.900672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.902660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.903257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.903533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.903801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.904436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:39.904755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.906873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.921488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 786/1573] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:39.922630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.932794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2413-797c-4784-ab62-16054e5694cf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.938122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.938789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.939071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.939474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.940256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.940371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 788/1574] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.942459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.943556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.972084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 787/1575] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:39.972905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.011153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7406e-c268-4b42-ac1a-36b81ac9d236 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.034830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507eed-1863-4fef-b414-e2d96a482382 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.039489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.040103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.040381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.040651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.041441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.043434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.067879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.068526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.068813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.069090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.069722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.071499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 788/1576] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.071668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.072326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.081964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.101377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 789/1577] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.102305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.129561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.143665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a309db-08c9-47dd-9c6c-3f4d223809a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.165573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a014-26dc-412b-9279-6305d6f3d8b9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.170894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.172121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.172400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.172674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] PATH_INFO: `/v3/projects/3f7d8432bdc442a1bc9357ceb9a96c60/users/d26716afd68a4fcfbbc323be98ea43d7/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.187993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7d8432bdc442a1bc9357ceb9a96c60, user_id=d26716afd68a4fcfbbc323be98ea43d7, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.189038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:40.189038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:42:40.189038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:42:40.191043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.201507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.202726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.203007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.203279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.203919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.205503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.206303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 790/1578] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:40 2022] PUT /identity/v3/projects/3f7d8432bdc442a1bc9357ceb9a96c60/users/d26716afd68a4fcfbbc323be98ea43d7/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.207736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.215781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 789/1579] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.216611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.230992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a9d4-b001-41bb-b5c5-f321b7c203aa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.277122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747406-cdee-41b8-ab90-227e5dc7294d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.282629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.283291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.283576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.283851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] PATH_INFO: `/v3/projects/a462c45b3b86441cb2e3c91f35ab8878/users/0b6ab231db9f4dca9acc01738fc1c639/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.288924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.289537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.289940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.290221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.290901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.292869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.298816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a462c45b3b86441cb2e3c91f35ab8878, user_id=0b6ab231db9f4dca9acc01738fc1c639, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.301483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.319669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 791/1580] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:40 2022] PUT /identity/v3/projects/a462c45b3b86441cb2e3c91f35ab8878/users/0b6ab231db9f4dca9acc01738fc1c639/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.320690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.326648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 790/1581] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.328687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.387229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.392626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8cb8-d98c-4f27-abf5-998f83815cfd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.398786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.399390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.399665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.399933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.400386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2427-a385-4797-85d1-7aedeb9d9116 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.400560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.402543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.405077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.406237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.406556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.406830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.407455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.409360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.426587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 792/1582] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.428056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.431270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 791/1583] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.431974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.439729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bf360-e25b-45c5-b951-1c92cb64608c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.442002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339d758-294f-4331-99de-8124e3656f7b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.444393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.445052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.445337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.446334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.446334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.448267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.448578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.449186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.449465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.449746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.450461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.452927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08982df4-c4bd-42d2-b2d3-52d78c331fe0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.463035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.474329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 793/1584] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.475416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.478195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 792/1585] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.479454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.479821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.480019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.491225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebca71e-3bb7-4c6e-85f3-76351173ca58 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.495997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.496637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.496915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.497188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.497840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.499505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.509060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] MFA Rules not processed for user `d26716afd68a4fcfbbc323be98ea43d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:40.510257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 794/1586] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.510390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.511084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.523746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432954-3508-4dac-8416-89f10532a093 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.528662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.529393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.529865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.530213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.531520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.533583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403ef01-8695-4e8f-a2fc-b6de42508409 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.555184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 795/1587] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:40.556492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.556778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.557055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.584693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 793/1588] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:40.585538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.588075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] MFA Rules not processed for user `0b6ab231db9f4dca9acc01738fc1c639`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:40.589603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.596862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697f6ac-60b2-4942-ac3a-4c7950f0d7c5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.601406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.602006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.602283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.602594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.603236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.604826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.616505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 794/1589] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.617739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.629315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0090-f415-4e68-a46f-1385fdd612ab admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.634850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.635508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.635795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.636067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.636766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.638609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.649802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 795/1590] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.650671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.660548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c88e3-7a12-49f6-a23f-9bd418aadabc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.665114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.665702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.665982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.666311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.666994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.668354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 796/1591] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:40.668518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.668614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.679891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 796/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.680698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.683880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090746b-00f8-4d42-8a2a-0845f7b6f65e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.689422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.690113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.690491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.690832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.691100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5d974-0f9f-4cdd-b597-f6667c25f243 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.691614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.693289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.696327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.696977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.697259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.697535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] PATH_INFO: `/v3/projects/524d39d780b2426cbacde028e1b061de/users/76dc93ac31604022a522b47559ab31b5/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.704279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 797/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.705187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.714067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524d39d780b2426cbacde028e1b061de, user_id=76dc93ac31604022a522b47559ab31b5, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.716858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e9c1f-147a-4ad3-bc1a-81aeb2fe1364 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.718665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776c3a2-9bbc-4b10-81e6-b0e60f64696a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.724781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.725435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.725821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.726128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.726808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.728407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.731848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 797/1594] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:40 2022] PUT /identity/v3/projects/524d39d780b2426cbacde028e1b061de/users/76dc93ac31604022a522b47559ab31b5/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.732993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d430-a33b-46e6-bc52-77e332180bbe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.733927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d430-a33b-46e6-bc52-77e332180bbe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.734332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d430-a33b-46e6-bc52-77e332180bbe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.735806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 798/1595] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:40.736589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69d430-a33b-46e6-bc52-77e332180bbe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.744958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 798/1596] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.744958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.752383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994134-fd79-47fb-bcc0-62fc024535f6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.806443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69d430-a33b-46e6-bc52-77e332180bbe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.812187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.812831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.813065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.813332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.813979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.815647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.816412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.817026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.817330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.817654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.818300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.819948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0902ab-baec-4670-aa76-a9f507806cfc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.825996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 799/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.826851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.833160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 799/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.834389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407421f0-ceed-4654-bb6c-f1cad026b27b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.834690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407421f0-ceed-4654-bb6c-f1cad026b27b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.834963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407421f0-ceed-4654-bb6c-f1cad026b27b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.836204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 800/1599] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:42:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:40.836823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407421f0-ceed-4654-bb6c-f1cad026b27b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.840919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065294-521e-486a-99d7-e1e0eacb1a5a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.843360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.845956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.846761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.847045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.847323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.847973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.849726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f19f5-fa97-4510-8a8a-4a157159e117 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.850115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407421f0-ceed-4654-bb6c-f1cad026b27b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.855927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.856512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.856788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.857209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.857843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.859494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fa90e-d642-42bf-9e81-77fc820994aa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.860064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 800/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.861335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.861631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.861906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.870266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 801/1601] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.871401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf46f-32c1-481f-a5fb-ab1c5e9bfecb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.871684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf46f-32c1-481f-a5fb-ab1c5e9bfecb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.871955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf46f-32c1-481f-a5fb-ab1c5e9bfecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.883613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:40.884843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.895895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 801/1602] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:40.897446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.902683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.908848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbe144-7688-4b39-b611-309cafb0f7ca None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.914085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.914740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.915020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.915293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.915933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.917532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.919523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6eeaf46f-32c1-481f-a5fb-ab1c5e9bfecb None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:40.920770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6eeaf46f-32c1-481f-a5fb-ab1c5e9bfecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.927890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 802/1603] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:40.928652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.938846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0540eb-a715-4f9a-8a9d-da0dfb30c179 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.943484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.944248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.944573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.944906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] PATH_INFO: `/v3/projects/19e5b7f483944ecfa24223586c5a35b6/users/e661e64b0d454f1e8f36114f4e34ccd0/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.960173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e5b7f483944ecfa24223586c5a35b6, user_id=e661e64b0d454f1e8f36114f4e34ccd0, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.962858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.976133 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 803/1604] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:40 2022] PUT /identity/v3/projects/19e5b7f483944ecfa24223586c5a35b6/users/e661e64b0d454f1e8f36114f4e34ccd0/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.976980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.995275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 802/1605] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:40.996888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.997024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.997278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.017089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.018396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.050811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238cd8-1b60-49a8-b986-221d26b36549 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.055380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.056026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.056301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.056569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] PATH_INFO: `/v3/projects/1a1f2b35a3df4b91930a2add2ca240fc/users/1404e3b072444e7aa688a89e8e4fa266/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.071945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1f2b35a3df4b91930a2add2ca240fc, user_id=1404e3b072444e7aa688a89e8e4fa266, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.074784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.084588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 803/1606] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.085412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.090080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 804/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:40 2022] PUT /identity/v3/projects/1a1f2b35a3df4b91930a2add2ca240fc/users/1404e3b072444e7aa688a89e8e4fa266/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.096615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.096806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b019c88-2b0f-442d-b2de-c0f24be75861 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.154144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.154839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.155135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.155436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] PATH_INFO: `/v3/projects/03e570df30ec409bae5b536144353a81/users/1c7906d5da9444dc86c4cd7904d54dc0/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.158358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.161072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c840cd-f672-4187-a293-f21bfceff325 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.165920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.166640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.166950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.167230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] PATH_INFO: `/v3/projects/604127b144fe4b43be3f5a4a4739d51e/users/ba878cb135d3497f8404d3a7751568ff/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.171617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e570df30ec409bae5b536144353a81, user_id=1c7906d5da9444dc86c4cd7904d54dc0, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.174267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.184706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604127b144fe4b43be3f5a4a4739d51e, user_id=ba878cb135d3497f8404d3a7751568ff, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.187996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.188116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 804/1608] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:41 2022] PUT /identity/v3/projects/03e570df30ec409bae5b536144353a81/users/1c7906d5da9444dc86c4cd7904d54dc0/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.195419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.195419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.207454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 805/1609] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:41 2022] PUT /identity/v3/projects/604127b144fe4b43be3f5a4a4739d51e/users/ba878cb135d3497f8404d3a7751568ff/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.208334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.242461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.250813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd736c-75b0-46d6-b581-79d27dbcdf83 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.274581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8627fd8-8762-45d0-8c04-0db1dd669ff8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.279594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.280347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.280548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.280905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] PATH_INFO: `/v3/projects/22d9fd2a9bec45938dd18540b1334a3b/users/d8581d46699041d5a4dcb8a43034863e/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.295470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d9fd2a9bec45938dd18540b1334a3b, user_id=d8581d46699041d5a4dcb8a43034863e, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.298663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912893c4-ed8d-4ea6-99b6-a2754bdd1585 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.302222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.302894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.303168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.303439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.312810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 806/1610] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:41 2022] PUT /identity/v3/projects/22d9fd2a9bec45938dd18540b1334a3b/users/d8581d46699041d5a4dcb8a43034863e/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.316651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.316784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.316784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.316784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.317042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.325159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 805/1611] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:41.326028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.345478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] MFA Rules not processed for user `e661e64b0d454f1e8f36114f4e34ccd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.347242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.425733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 807/1612] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.425733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.427539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212544-7c58-4357-866c-3dfd1f1dfc63 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.475823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.505090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.505702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.506036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.506244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.522855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84455121-2b68-47d6-830a-d845d1a7062f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.529534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.530949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.531581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.531581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.572935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.574640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f41a24-395b-4e75-b9b1-d005efd689bf service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.579912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.590654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 806/1613] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:41.592168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ebf7-9c26-4817-978a-1169c560ccff None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.592459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ebf7-9c26-4817-978a-1169c560ccff None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.593298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ebf7-9c26-4817-978a-1169c560ccff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.605242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.607683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f89bd-837b-4f5a-9be8-7831540f2f49 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.621493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 808/1614] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:41.624121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34943-55aa-440d-90a3-58fa4dae0f0b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.624405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34943-55aa-440d-90a3-58fa4dae0f0b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.624679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34943-55aa-440d-90a3-58fa4dae0f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.625030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8193ebf7-9c26-4817-978a-1169c560ccff None None] MFA Rules not processed for user `1404e3b072444e7aa688a89e8e4fa266`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.626565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8193ebf7-9c26-4817-978a-1169c560ccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.670321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-06b34943-55aa-440d-90a3-58fa4dae0f0b None None] MFA Rules not processed for user `1c7906d5da9444dc86c4cd7904d54dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.673614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06b34943-55aa-440d-90a3-58fa4dae0f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.724769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 807/1615] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.726788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.727133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.727414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.766984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.768973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.801243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 809/1616] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.802679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.802953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.803222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.832940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.834878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.920928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 810/1617] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.922329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.941363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 808/1618] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.943708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d3b84-46d6-45be-8be8-202c9a2591c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.943878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.958814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.959704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.960313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.960868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.961705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.961928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fec06a-46c6-47ff-9d76-0a9f8564770d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.964325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.969667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.970291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.970687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.970878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.971829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.973640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010d0ed-aacd-4a9d-8b20-dabb5e3fe641 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.979452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:42.005473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 809/1619] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.009464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 811/1620] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.009686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.010367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4051-23e1-43d1-b30e-eda0ed3f8f9e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.010367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4051-23e1-43d1-b30e-eda0ed3f8f9e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.010870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4051-23e1-43d1-b30e-eda0ed3f8f9e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.012449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 810/1621] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 15:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:42.013327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6d77-fb68-4f7f-bce7-225d4369eb79 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.013829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6d77-fb68-4f7f-bce7-225d4369eb79 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.014117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6d77-fb68-4f7f-bce7-225d4369eb79 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.016143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 811/1622] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 15:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:42.016976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa6d77-fb68-4f7f-bce7-225d4369eb79 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.078709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80886-972a-46e9-b2a9-1358e1edad7d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.083436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.084054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.084331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.084533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa6d77-fb68-4f7f-bce7-225d4369eb79 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.085169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.088698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.089427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.089899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.090241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.090383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:42.169721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.171469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.172928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.173308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.183982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 812/1623] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:42.184833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.185636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 812/1624] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:42.186385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.195064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2763d-cf2b-4dc8-acdb-826464481c81 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.200249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.201076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.201251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.201688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.208891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff44-80f6-42d3-b751-4c854ed635c7 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.212929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.216614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.217317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.218148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.219201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.219335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2a9c-f796-4b31-9fb5-8d768072c29c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.219898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.221989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a178f1-f2c2-4d90-b2bf-eb6d37ebc69e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.255586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 813/1625] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/users => generated 299 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.262587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.262587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.262587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.283658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 813/1626] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/users => generated 306 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.285900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.286246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.286590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.304187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:42.305422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:42.312268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:42.314655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:42.427536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 814/1627] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:42.428823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.446861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be5c54-7e16-4219-8b8f-116c2139a9ee None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.454680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.455556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.456051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.456330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.457253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.459862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.467409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:42.468989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 814/1628] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:42.470163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.479351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 815/1629] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:42.481108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.490101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3027d-30a3-41fb-80fe-38f439b07b24 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.496549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.497338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.497739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.498116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.499567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.501354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.517728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 815/1630] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:42.518733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.531876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b68b-692c-46dd-8ebd-919b4c079688 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.538308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.539600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.539949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.540368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] PATH_INFO: `/v3/projects/29a41da4effe466484031057fa385b30/users/18bbd7ca3e1c43d6890fe2db932a4a2f/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.563078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a41da4effe466484031057fa385b30, user_id=18bbd7ca3e1c43d6890fe2db932a4a2f, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.563372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3efd7-364e-4977-8c36-8cfa6881f834 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.566712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.567730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.568582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.569018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.569435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.583255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.585166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.599229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 816/1632] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:42.600324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 816/1632] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:42 2022] PUT /identity/v3/projects/29a41da4effe466484031057fa385b30/users/18bbd7ca3e1c43d6890fe2db932a4a2f/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:42.600823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.601321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.611430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce94e-38ce-4abb-9284-37992362839a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.671172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.671854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.672233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.673180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.689270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:42.724197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0bf2-7cb5-4b52-b952-4e4a26924f13 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.733723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.735098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.735379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.735650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] PATH_INFO: `/v3/projects/b4afdf975268425588e7d7f4058436e5/users/9675b9a1e0da4d31b7fbe96b0abb1801/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.769656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4afdf975268425588e7d7f4058436e5, user_id=9675b9a1e0da4d31b7fbe96b0abb1801, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.775984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.806326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 817/1633] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:42 2022] PUT /identity/v3/projects/b4afdf975268425588e7d7f4058436e5/users/9675b9a1e0da4d31b7fbe96b0abb1801/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:42.807503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.844505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.849601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.876345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 817/1634] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:42.877238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.903322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9af72-d211-4a36-b118-6f07575a0ff8 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.940752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda31d8-303e-48d1-88db-d10767b78b62 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.956596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.957229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.957639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.957922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.958603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.962586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.988338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 818/1635] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:42.989266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.994521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:43.015214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542528-23fa-4e5a-a27f-dec139ae90a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:43.029068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.029746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.030067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.030857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] PATH_INFO: `/v3/projects/29a41da4effe466484031057fa385b30/users/18bbd7ca3e1c43d6890fe2db932a4a2f/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.035098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a41da4effe466484031057fa385b30, user_id=18bbd7ca3e1c43d6890fe2db932a4a2f, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:43.037749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915c5b8-426b-4fdf-97c2-902a494ae2df admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:43.040049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.041887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.041887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.041887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.042458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:43.046576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:43.073297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 818/1637] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:43.075242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:43.077646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 819/1637] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:42 2022] PUT /identity/v3/projects/29a41da4effe466484031057fa385b30/users/18bbd7ca3e1c43d6890fe2db932a4a2f/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:43.080306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.082037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.082318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.096830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cdec4-946c-4acd-99f7-e91b786714cd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:43.151398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] MFA Rules not processed for user `18bbd7ca3e1c43d6890fe2db932a4a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:43.155043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:43.251024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.251817 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.252299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.252687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] PATH_INFO: `/v3/projects/b4afdf975268425588e7d7f4058436e5/users/9675b9a1e0da4d31b7fbe96b0abb1801/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.269386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4afdf975268425588e7d7f4058436e5, user_id=9675b9a1e0da4d31b7fbe96b0abb1801, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:43.277264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abb329-384b-4a3d-89a4-16434f78bbea admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:43.313615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 819/1638] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:43 2022] PUT /identity/v3/projects/b4afdf975268425588e7d7f4058436e5/users/9675b9a1e0da4d31b7fbe96b0abb1801/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:43.315915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.320366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.320761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.328490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 820/1639] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:43.411136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] MFA Rules not processed for user `9675b9a1e0da4d31b7fbe96b0abb1801`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:43.412714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:43.600845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 820/1640] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:43.692431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:43.787265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:43.836543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:43.846280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabfd56-47be-48ac-879b-67724772cd90 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:43.985836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.986447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.986820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.987010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.006844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e07c5-63bf-4ded-a6d1-fd847fdb90d4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.084297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.084297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.084297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.084297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.094174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.094174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f108eab-1f9b-475a-a29c-8bf91aceaa78 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.118116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 821/1641] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:44.184113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.187934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.205492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 821/1642] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:44.210239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.210604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.210879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.221408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:44.294302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] MFA Rules not processed for user `ba878cb135d3497f8404d3a7751568ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:44.296561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:44.348195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.504170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 822/1643] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:44 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:44.505278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.520725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec8ef-10d0-4489-9f35-63db7d116227 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.526854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5610-4ade-42cc-ae74-bc9a7c294c94 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.532553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.533765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.534254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.534664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.536879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.537766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.538261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.538867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.594284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:44.623381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.626711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.629733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.632292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.638946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 823/1644] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:44.640609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.691237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 822/1645] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:44.692948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.738664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dbeb2-6467-4d2f-86f0-543fa4be5761 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.751435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.754262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.759760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.759760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.762280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.767461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.848245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 824/1646] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:44 2022] POST /identity/v3/projects => generated 405 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:44.857393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.879265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62badd-5eb0-416a-a557-6009d0710e34 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.908089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.003001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b0a9-694e-4d5d-a810-6eedf5fcae65 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.010743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.011353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.011761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.011943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] PATH_INFO: `/v3/users/e661e64b0d454f1e8f36114f4e34ccd0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.020309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e661e64b0d454f1e8f36114f4e34ccd0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.020309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.023098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.023098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.023098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.023098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.024603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.025095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.112049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 825/1647] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:44 2022] POST /identity/v3/users => generated 309 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:45.112049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.144076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f018e-a765-4659-b7b2-68b73ba5047f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.148605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e661e64b0d454f1e8f36114f4e34ccd0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.161347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.162731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.164371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.164769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.164872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e661e64b0d454f1e8f36114f4e34ccd0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.165560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e661e64b0d454f1e8f36114f4e34ccd0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.166198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e661e64b0d454f1e8f36114f4e34ccd0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.166359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.172499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.182696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e661e64b0d454f1e8f36114f4e34ccd0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.207164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 826/1648] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:45.209763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.211648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.214657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.218003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 823/1649] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:44 2022] DELETE /identity/v3/users/e661e64b0d454f1e8f36114f4e34ccd0 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:45.219601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.225337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.239646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8ea02-11b2-4453-a268-2be4d843c055 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.409308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b85a-ee17-4631-8470-51e9f85fae27 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.433881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.435530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.435948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.437111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] PATH_INFO: `/v3/projects/6a382126240d4d15a681c1c62a263175/users/4e3c8b6fb2f646fd9da80371685c3281/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.479331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a382126240d4d15a681c1c62a263175, user_id=4e3c8b6fb2f646fd9da80371685c3281, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.479932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.481017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.481333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.481603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] PATH_INFO: `/v3/users/1c7906d5da9444dc86c4cd7904d54dc0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.482821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7906d5da9444dc86c4cd7904d54dc0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.484642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9bf11-c6ca-47ae-9934-cf02d7241cc3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.486613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:45.486613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:42:45.486613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: Sep 28 15:42:45.489916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.511682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 824/1650] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:45 2022] PUT /identity/v3/projects/6a382126240d4d15a681c1c62a263175/users/4e3c8b6fb2f646fd9da80371685c3281/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:45.515946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.516238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.516523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.565231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:45.569251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.616072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7906d5da9444dc86c4cd7904d54dc0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.630048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7906d5da9444dc86c4cd7904d54dc0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.631323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7906d5da9444dc86c4cd7904d54dc0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.634032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7906d5da9444dc86c4cd7904d54dc0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.653168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb827978-4c41-49bf-a1ab-bbeddba75c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7906d5da9444dc86c4cd7904d54dc0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.688635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 825/1651] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:45.689562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.690956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.693770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 827/1652] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:45 2022] DELETE /identity/v3/users/1c7906d5da9444dc86c4cd7904d54dc0 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:45.699668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.700293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.700823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.705388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d24dd8-306c-4171-bf40-62ded73d5efa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.740736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:45.743806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.774212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.774827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.775094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.775385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.781220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.783917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.795854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.819217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 826/1653] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:45.820245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.822106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.920001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.944856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 828/1654] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:45.949041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.986970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa461e1-9621-4483-9b4c-89fb9d1e375c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.002603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.003127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.003470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.003866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] PATH_INFO: `/v3/projects/6a382126240d4d15a681c1c62a263175/users/4e3c8b6fb2f646fd9da80371685c3281/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.009700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a382126240d4d15a681c1c62a263175, user_id=4e3c8b6fb2f646fd9da80371685c3281, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.017927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.041139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fada-c344-4981-bcd5-9a7920e0b333 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.045313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.046806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.047255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.047815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.059895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.062141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.074453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 829/1655] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:45 2022] PUT /identity/v3/projects/6a382126240d4d15a681c1c62a263175/users/4e3c8b6fb2f646fd9da80371685c3281/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:46.078242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.079650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 827/1656] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:46.205706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595c2b-96b0-41c1-83ac-043ddbeed6dd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.213323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.214488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.214872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.227102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.360250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.361720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735a142-6912-41a2-9106-23dda6b589b5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.383600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 830/1657] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:47.339228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.341614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.361499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cc4d-bf62-4927-9359-1f152f028b7f service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.402164 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.402837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.403435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.403914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] PATH_INFO: `/v3/users/0b6ab231db9f4dca9acc01738fc1c639` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.405949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6ab231db9f4dca9acc01738fc1c639)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.410559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.490910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6ab231db9f4dca9acc01738fc1c639'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.499221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6ab231db9f4dca9acc01738fc1c639'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.500159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6ab231db9f4dca9acc01738fc1c639'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.500893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6ab231db9f4dca9acc01738fc1c639'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.515446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6ab231db9f4dca9acc01738fc1c639'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.550816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 828/1658] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:47 2022] DELETE /identity/v3/users/0b6ab231db9f4dca9acc01738fc1c639 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:47.557391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.558134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.559250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.639931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:47.641574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.725673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:47.759716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 831/1659] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:47.772100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.774008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.914262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fc1b1-7257-4ad7-bc84-2e1478d95801 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.918105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.918758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.919046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.919322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.930178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.932231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.949753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 829/1660] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:47.961508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.963285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.981312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c83a-28c7-4f41-8719-e8cb23c66b32 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.018445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.020455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.022132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.022871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] PATH_INFO: `/v3/projects/19e5b7f483944ecfa24223586c5a35b6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.026742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e5b7f483944ecfa24223586c5a35b6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.040684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.124196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.125714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:48.177660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b148da-785f-493b-9230-b708a7b02378 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.195954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e5b7f483944ecfa24223586c5a35b6', 'request_id': 'req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.223644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e5b7f483944ecfa24223586c5a35b6', 'request_id': 'req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.253761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e5b7f483944ecfa24223586c5a35b6', 'request_id': 'req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.296562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] The token cache is being invalidate because project 19e5b7f483944ecfa24223586c5a35b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:48.299670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.300615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.305704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 832/1661] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:47 2022] DELETE /identity/v3/projects/19e5b7f483944ecfa24223586c5a35b6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:48.307263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.309303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:48.319849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:48.394556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.395929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.396637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.397217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] PATH_INFO: `/v3/projects/03e570df30ec409bae5b536144353a81` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.398618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e570df30ec409bae5b536144353a81)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.400938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:42:48.400938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:42:48.400938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: Sep 28 15:42:48.402716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.473330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07a91e-0b3c-4e6b-ae00-c3c79e05c4a8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.479864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.480693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.481150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.481519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] PATH_INFO: `/v3/users/18bbd7ca3e1c43d6890fe2db932a4a2f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.483425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bbd7ca3e1c43d6890fe2db932a4a2f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.485798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.597872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bbd7ca3e1c43d6890fe2db932a4a2f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.601631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e570df30ec409bae5b536144353a81', 'request_id': 'req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.611822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bbd7ca3e1c43d6890fe2db932a4a2f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.624966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bbd7ca3e1c43d6890fe2db932a4a2f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.631341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bbd7ca3e1c43d6890fe2db932a4a2f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.632342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e570df30ec409bae5b536144353a81', 'request_id': 'req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.657551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3df1b16c-52d3-4ccb-b17d-c322310d4de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bbd7ca3e1c43d6890fe2db932a4a2f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.663343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e570df30ec409bae5b536144353a81', 'request_id': 'req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.685135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:48.703800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 833/1662] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:48 2022] DELETE /identity/v3/users/18bbd7ca3e1c43d6890fe2db932a4a2f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:48.739824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] The token cache is being invalidate because project 03e570df30ec409bae5b536144353a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:48.740466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.741378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-60ab7b60-a3c3-435c-a4d6-212e1e99c19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:48.746546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 830/1663] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:48 2022] DELETE /identity/v3/projects/03e570df30ec409bae5b536144353a81 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:48.751955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.752488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.752488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.800658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:48.801997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.021348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 834/1664] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:49.023303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.023805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.024108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.065953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:49.069356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.085221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.091337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 831/1665] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:49.097525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b93f-1711-4c86-b396-6a094f6428b5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.103982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.104547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.104894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.105611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.106134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:49.106549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.108494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.140053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.147424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 835/1666] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:49.169891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1cae2-8e0d-44ff-bcba-058bb1b480ca None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.186737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.188226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.189062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.190688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.192583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.199144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.258906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 832/1667] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:49 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:49.259621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.275352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bc2d6-a276-47df-8a1e-ba88b51ae947 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.472955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.477078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.513760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.516373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.516656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.516929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.519169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.526789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.599583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 836/1668] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:49 2022] POST /identity/v3/users => generated 287 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:49.604470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.641629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7ebd-dee6-4af4-a595-a83868644cd2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.672928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.673803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.674970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.675253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.676329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.680060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c1088-63a5-4b1f-8744-fb1bb3713654 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.687979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8fb4-df23-4a1f-8823-b59cf4f416fb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.720856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 837/1669] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:49.721043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.721937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.722312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.722769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] PATH_INFO: `/v3/projects/a462c45b3b86441cb2e3c91f35ab8878` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.723233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c676e-7589-4c3c-9f9f-f751d5a25e4f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.724177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c676e-7589-4c3c-9f9f-f751d5a25e4f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.724598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c676e-7589-4c3c-9f9f-f751d5a25e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.725417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a462c45b3b86441cb2e3c91f35ab8878)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.727487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.782752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-557c676e-7589-4c3c-9f9f-f751d5a25e4f None None] MFA Rules not processed for user `4e3c8b6fb2f646fd9da80371685c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:49.783539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-557c676e-7589-4c3c-9f9f-f751d5a25e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.818851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:49.834544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a462c45b3b86441cb2e3c91f35ab8878', 'request_id': 'req-1c2811c9-1bf1-4583-8a4e-5662cd493d83'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:49.851074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a462c45b3b86441cb2e3c91f35ab8878', 'request_id': 'req-1c2811c9-1bf1-4583-8a4e-5662cd493d83'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:49.874284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a462c45b3b86441cb2e3c91f35ab8878', 'request_id': 'req-1c2811c9-1bf1-4583-8a4e-5662cd493d83'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:49.891405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:49.899565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] The token cache is being invalidate because project a462c45b3b86441cb2e3c91f35ab8878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:49.899971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:49.900697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:49.903794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 833/1670] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:49 2022] DELETE /identity/v3/projects/a462c45b3b86441cb2e3c91f35ab8878 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:49.905619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.909810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.958137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 838/1671] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:49.959634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.959920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.960194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.007649 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:50.009183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.047677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2811c9-1bf1-4583-8a4e-5662cd493d83 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.057099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.057728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.058368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.058938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.059599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.064038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.076401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 834/1672] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:50.077321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.079102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.097204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34d0d2-e43e-43cf-a86e-6db2cf29ba92 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.113233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.114161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.114843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.115290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] PATH_INFO: `/v3/projects/29a41da4effe466484031057fa385b30` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.117383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a41da4effe466484031057fa385b30)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.119749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.164738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 839/1673] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:50.165629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.191298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb772-ac53-4160-b120-24c7c5c4fe5d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.212114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a41da4effe466484031057fa385b30', 'request_id': 'req-dd275063-9276-4257-a537-dc49d659be0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:50.228068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a41da4effe466484031057fa385b30', 'request_id': 'req-dd275063-9276-4257-a537-dc49d659be0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:50.241805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a41da4effe466484031057fa385b30', 'request_id': 'req-dd275063-9276-4257-a537-dc49d659be0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:50.266510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] The token cache is being invalidate because project 29a41da4effe466484031057fa385b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:50.266806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:50.268150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd275063-9276-4257-a537-dc49d659be0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:50.277019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 835/1674] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:50 2022] DELETE /identity/v3/projects/29a41da4effe466484031057fa385b30 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:50.277019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d059f-3f2e-4f87-9494-7e4a03e8635d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.277019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d059f-3f2e-4f87-9494-7e4a03e8635d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.277019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d059f-3f2e-4f87-9494-7e4a03e8635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.320335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e5d059f-3f2e-4f87-9494-7e4a03e8635d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:50.321879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e5d059f-3f2e-4f87-9494-7e4a03e8635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.337589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.338286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.338597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.338870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] PATH_INFO: `/v3/projects/751040822a3d462a8443274b80a92e40/users/d58840e417b641409e00df0771215b75/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.355551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751040822a3d462a8443274b80a92e40, user_id=d58840e417b641409e00df0771215b75, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.358717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aedfd8-0cef-4723-8be0-eb316069f660 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.376373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 840/1675] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:50 2022] PUT /identity/v3/projects/751040822a3d462a8443274b80a92e40/users/d58840e417b641409e00df0771215b75/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:50.378105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.378574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.378964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.408916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:50.437573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:50.438903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.442301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 836/1676] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:50.444017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.444466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.444877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.556859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] MFA Rules not processed for user `d58840e417b641409e00df0771215b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:50.557632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.598281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 841/1677] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:50.600099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.620736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbdd8d-be58-4cb6-88ec-ecf56991cafd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.635719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.636487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.636764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.638992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.641975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:50.642374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.644152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346322a-9a36-46ad-870d-455dcfc69b20 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.651607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:50.676466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 842/1678] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:50.677879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.678164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.678446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.713761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 837/1679] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:50.714811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.715709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:50.717976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.740954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90a13e-16d3-44d2-b160-759aaffd512f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.745581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 843/1680] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:50.756748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.758084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.758430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.758721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.759570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.760103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.761833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:50.763841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.787709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33a232-bb93-460d-a508-edb2a1ea0960 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.803395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.804029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.804309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.804588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.809955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.812808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:50.816343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 838/1681] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:50.817327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.879172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 844/1682] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:50.879172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:50.954543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4b40-b910-4bd9-bd9b-57083f2372bc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.975232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00514-3327-425e-96cb-293e91bb62c8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:50.982256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:50.983507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:50.983790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:50.984060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:50.984780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:50.986731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05542218-6e94-4917-800b-632124b842c4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.023294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 845/1683] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:50 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:51.028012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.028301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.028581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.081494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.082170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.082398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.082918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.083686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:51.083994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.085616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.089446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] MFA Rules not processed for user `76dc93ac31604022a522b47559ab31b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:51.092214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:51.094786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:51.100898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 839/1684] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:51.104550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.126374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e02f40-ae07-4cf1-87a5-58b818a46550 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.135162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.135757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.136033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.136605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.136945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.139177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.170348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 840/1685] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:51 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:51.176111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.187967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391bce0-750f-4524-93b0-b086d817863e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.194366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.195020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.195298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.195570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.196240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.197869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.206906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 846/1686] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:51.221381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 841/1687] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:51.222339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.223106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.233552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721b74-8d06-4c35-a574-e8d81990ef27 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.239481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae247f-6be0-48f0-ba00-b85c22269db8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.250311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.251170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.251542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.251907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.252821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.254564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.254925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.255552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.255834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.256109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.256756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.266015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e8654-edc5-4e33-b78b-c1a6733ec772 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.278596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 842/1688] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:51.279502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.301056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dde4e-4ba9-497f-8d7d-c663ac3aa183 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.307388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 847/1689] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:51 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:51.307388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.307388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.307388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.316231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.317015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.317283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.317616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.318330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.320117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.336062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] MFA Rules not processed for user `76dc93ac31604022a522b47559ab31b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:51.337424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:51.344843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 843/1690] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:51.347239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.367334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:51.368804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80406283-8a72-468a-a3f2-501ca56aeebc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.441242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 848/1691] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:51.442097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.479360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.480047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.480329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.480608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] PATH_INFO: `/v3/projects/a5367e5b8d6d45739becaaf576bb346c/users/090e239fc58949a3ac48e0eff6aafb8b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.503398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5367e5b8d6d45739becaaf576bb346c, user_id=090e239fc58949a3ac48e0eff6aafb8b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.505495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.531017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191928c7-cb1e-4c2e-b794-645ab3cf678a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.533541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 844/1692] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:51 2022] PUT /identity/v3/projects/a5367e5b8d6d45739becaaf576bb346c/users/090e239fc58949a3ac48e0eff6aafb8b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:51.534566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.656627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.657255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.657538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.658175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.658901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.661135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.663969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb22951-69f8-463a-aa3b-3100ba432ebf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.679480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.680357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.680817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.681185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] PATH_INFO: `/v3/projects/ad8d57beb105473ea99ee1f3919aa04a/users/c9e29a792d1f4f038f0b975a5f84e869/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.707579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 849/1693] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:51 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:51.708446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.714209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:51.714609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8d57beb105473ea99ee1f3919aa04a, user_id=c9e29a792d1f4f038f0b975a5f84e869, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.725030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.770356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 845/1694] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:51 2022] PUT /identity/v3/projects/ad8d57beb105473ea99ee1f3919aa04a/users/c9e29a792d1f4f038f0b975a5f84e869/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:51.770356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.793677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:51.892877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:51.904442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc38ca-9f84-4a21-9f23-7655d46e63a5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.909514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6dab3-e81d-4b4a-808d-e9bcadce859c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.924886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.926824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.927120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.927397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.928072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.929718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfab05d-2469-4fa2-8a34-9c12cb955187 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.945169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 846/1695] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:51.946839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067cb72-0adf-45f1-88fc-f0b6ac36694c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.947236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067cb72-0adf-45f1-88fc-f0b6ac36694c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.947520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067cb72-0adf-45f1-88fc-f0b6ac36694c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.948901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 847/1696] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:51.949663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067cb72-0adf-45f1-88fc-f0b6ac36694c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.964587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067cb72-0adf-45f1-88fc-f0b6ac36694c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.976846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.977586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.977862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.978134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.978968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.980609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e669-84ee-444d-b071-22e49dfb34e6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.006273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 848/1697] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:52.010622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec831c-77b5-463c-bf0a-8aa9b7762f7a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.011012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec831c-77b5-463c-bf0a-8aa9b7762f7a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.011295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec831c-77b5-463c-bf0a-8aa9b7762f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.023290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.024080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.024496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.024861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.078705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6eec831c-77b5-463c-bf0a-8aa9b7762f7a None None] MFA Rules not processed for user `c9e29a792d1f4f038f0b975a5f84e869`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:52.079936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6eec831c-77b5-463c-bf0a-8aa9b7762f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:52.096721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.098712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.119046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 850/1698] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:52.122635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.151733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54416fdd-a941-484f-b4c8-dda5605c7d29 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.164254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.165101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.165384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.165659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] PATH_INFO: `/v3/projects/a5367e5b8d6d45739becaaf576bb346c/users/090e239fc58949a3ac48e0eff6aafb8b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.167807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5367e5b8d6d45739becaaf576bb346c, user_id=090e239fc58949a3ac48e0eff6aafb8b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.170574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.205717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 851/1699] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:52 2022] PUT /identity/v3/projects/a5367e5b8d6d45739becaaf576bb346c/users/090e239fc58949a3ac48e0eff6aafb8b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:52.206464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.236456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 849/1700] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:52.237766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7628-a7b3-4c6c-8f88-2b65bec4d79d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.238053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7628-a7b3-4c6c-8f88-2b65bec4d79d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.238336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7628-a7b3-4c6c-8f88-2b65bec4d79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.306673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9eea7628-a7b3-4c6c-8f88-2b65bec4d79d None None] MFA Rules not processed for user `090e239fc58949a3ac48e0eff6aafb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:52.307970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9eea7628-a7b3-4c6c-8f88-2b65bec4d79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:52.365246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3bbc-467e-4bd1-939b-ce059f58ab1a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.384494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.385124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.387226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.387528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.388197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.389847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.418610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 852/1701] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:52.425424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.457051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2ee0a-65e2-46e3-ac31-47e835c959e0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.468286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.468857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.469282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.469562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.470520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.475176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.500266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 850/1702] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:52.506035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.506330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.507976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.546404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 853/1703] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:52 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:52.547355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.552378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:52.558003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:52.559910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:52.585373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:52.692705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed1168-8b7b-4812-a7f0-02af9cd40de0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.713184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.713942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.714230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.714536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] PATH_INFO: `/v3/projects/b20b0ef06a1842bc890551a27f696999/users/36abb8abc44e438398b8927f70016a86/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.738086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 851/1704] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:52.739030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.741472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:52.759855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20b0ef06a1842bc890551a27f696999, user_id=36abb8abc44e438398b8927f70016a86, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.763650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:52.793339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 854/1705] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:52 2022] PUT /identity/v3/projects/b20b0ef06a1842bc890551a27f696999/users/36abb8abc44e438398b8927f70016a86/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:52.796658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:52.853322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2473e0-e0a5-4aa2-a25c-ba9a03a74ac3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.946021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c5d56-e31b-4cbf-9ea3-98490651655f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:52.954383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.955413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.955921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.956515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.958745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:52.959382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:52.959676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:52.959951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:52.960666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:52.962732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a12dfa-c465-426b-bbc0-fb02c01b8083 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.015559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 855/1706] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:52 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:53.019864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.020819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.023036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.102774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] MFA Rules not processed for user `36abb8abc44e438398b8927f70016a86`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:53.104248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:53.138397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.140963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.157261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 852/1707] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:53.159007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.160812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:53.188204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:53.231552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 856/1708] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:53.232786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.242059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:53.251995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6d4c7-09ad-4b2e-9958-e04da7759603 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.268464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.269900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.270387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.270835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.271644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.277530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.308363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 857/1709] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:53.309309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.325209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f55104-7592-429e-aaf2-0de00ce6d0c5 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.337894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.338253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09510-2a04-4e59-9025-7f201ac22870 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.339143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.339995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.340415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.348832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.349469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.349956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.350235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.350939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.352950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.358196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.360371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.391361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 853/1710] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:53.392693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.407654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591736a-01e0-4808-96a6-e2a6c3f6da9e service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.414770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.415845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.416227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.416508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] PATH_INFO: `/v3/projects/e2956355677a4f73b443955000ad4899/users/2101dd0b8a6146ba9887ba93002455ca/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.437929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 858/1711] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:53 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:53.441784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.477529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2956355677a4f73b443955000ad4899, user_id=2101dd0b8a6146ba9887ba93002455ca, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.487855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.511872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 854/1712] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:53 2022] PUT /identity/v3/projects/e2956355677a4f73b443955000ad4899/users/2101dd0b8a6146ba9887ba93002455ca/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:53.516615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.638797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415ffd3-8f2f-49ef-b386-83590cdcb223 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.807906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683857f-5bc2-4054-ad44-041ab0b6309b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.821992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.823811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.824297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.824695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.825437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.828983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda06a6c-5d46-46c0-9d20-bc0345144cda admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.845917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 855/1713] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:53.848073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329701bb-1a17-453f-ad35-475bed57d275 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.848367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329701bb-1a17-453f-ad35-475bed57d275 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.848694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329701bb-1a17-453f-ad35-475bed57d275 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.850771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 856/1714] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 15:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:53.851731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329701bb-1a17-453f-ad35-475bed57d275 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:53.866686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329701bb-1a17-453f-ad35-475bed57d275 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:53.868107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.871579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.871900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.872185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.873190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:53.874817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.875256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.875781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] PATH_INFO: `/v3/projects/e2956355677a4f73b443955000ad4899/users/2101dd0b8a6146ba9887ba93002455ca/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.877664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.879264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2956355677a4f73b443955000ad4899, user_id=2101dd0b8a6146ba9887ba93002455ca, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:53.883731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a215aa-5a18-437c-abaa-f452786aaf39 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.885419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe380f-72b1-4674-a24f-976bea25cdd3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:53.904605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 857/1715] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:53 2022] PUT /identity/v3/projects/e2956355677a4f73b443955000ad4899/users/2101dd0b8a6146ba9887ba93002455ca/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:53.912090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.912402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.916287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:53.948796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 859/1716] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:53 2022] POST /identity/v3/users => generated 301 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:53.949172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:53.949630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:53.952128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.035503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] MFA Rules not processed for user `2101dd0b8a6146ba9887ba93002455ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:54.036768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.094995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:54.097815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.242728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:54.276275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 860/1717] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:54.277245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.300455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:54.314254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 858/1718] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:54.316135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.319517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.430706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30786cd-26dd-4037-b3b2-6fadfdf746aa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:54.442279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:54.450836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e7d50-f8d1-49a4-ba29-fb12ee20c407 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:54.451010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:54.460481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:54.460799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.462090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:54.463790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:54.478540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:54.479219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:54.479501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:54.480318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] PATH_INFO: `/v3/users/ba878cb135d3497f8404d3a7751568ff` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.483397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba878cb135d3497f8404d3a7751568ff)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:54.485146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:54.496723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 861/1719] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:54.496723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.504125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.590206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba878cb135d3497f8404d3a7751568ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:54.607123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba878cb135d3497f8404d3a7751568ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:54.608525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba878cb135d3497f8404d3a7751568ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:54.609447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba878cb135d3497f8404d3a7751568ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:54.627401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba878cb135d3497f8404d3a7751568ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:54.649371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8f97-2a20-436a-9eb7-645d4b8c92e1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:54.659587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:54.662182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:54.662506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:54.662789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.675440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 859/1720] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:54 2022] DELETE /identity/v3/users/ba878cb135d3497f8404d3a7751568ff => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:54.676687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.681093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.682945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.774154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b44d-26b2-4504-bbb8-a6b21fd8e533 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:54.776241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:54.777906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:54.779281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:54.779949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:54.780232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:54.780500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.782271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.794204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 862/1721] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:54.795818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.797294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.804330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:54.805803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:54.815369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67145a-a88a-4dd1-a8f7-74e425be0b5a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:54.829720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 860/1722] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:54.830557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:54.831898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:54.885496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:54.892702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:54.941772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:54.942887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:54.943542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:54.944045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] PATH_INFO: `/v3/users/1404e3b072444e7aa688a89e8e4fa266` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:54.945933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1404e3b072444e7aa688a89e8e4fa266)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:54.949470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.004458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2b107-2f4b-4a28-8018-29642f0a3719 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:55.032756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:55.033864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.034513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.034954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] PATH_INFO: `/v3/projects/ca244458caa44504905f45879460eda8/users/a909a5aa60b643e8a8029e651f262ad6/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.080676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca244458caa44504905f45879460eda8, user_id=a909a5aa60b643e8a8029e651f262ad6, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:55.085188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.099396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1404e3b072444e7aa688a89e8e4fa266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.125872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1404e3b072444e7aa688a89e8e4fa266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.126783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1404e3b072444e7aa688a89e8e4fa266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.129718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1404e3b072444e7aa688a89e8e4fa266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.133777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 861/1723] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:54 2022] PUT /identity/v3/projects/ca244458caa44504905f45879460eda8/users/a909a5aa60b643e8a8029e651f262ad6/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:55.136241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:55.144427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1404e3b072444e7aa688a89e8e4fa266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.145380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.228038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 863/1724] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:54 2022] DELETE /identity/v3/users/1404e3b072444e7aa688a89e8e4fa266 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:55.229294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:55.237064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.399889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b7c72-2789-43fd-a421-14c65a66c48e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:55.405722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f29-276e-411c-ae3d-0827fa616898 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:55.415334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:55.416264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.416544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.416813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.419691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:55.426117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145658d1-acfd-4158-bc98-b44e1a07fde6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.428903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:55.431908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.432265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.432782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] PATH_INFO: `/v3/users/d58840e417b641409e00df0771215b75` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.437712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58840e417b641409e00df0771215b75)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:55.442882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.461736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 864/1725] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:55.463643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.463933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.465488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.536596 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:55.539707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.568866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 865/1726] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:55.570219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:55.612393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca171-6e7d-418d-a52b-d1a9b7f9aca1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:55.632086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:55.633222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.633680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.635423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] PATH_INFO: `/v3/projects/ca244458caa44504905f45879460eda8/users/a909a5aa60b643e8a8029e651f262ad6/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.640298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca244458caa44504905f45879460eda8, user_id=a909a5aa60b643e8a8029e651f262ad6, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:55.647449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a243633-5bcb-4564-9372-ce4b0c0dae23 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.651327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58840e417b641409e00df0771215b75'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.660061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58840e417b641409e00df0771215b75'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.661178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58840e417b641409e00df0771215b75'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.662018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58840e417b641409e00df0771215b75'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.679092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58840e417b641409e00df0771215b75'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:55.679302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 866/1727] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:55 2022] PUT /identity/v3/projects/ca244458caa44504905f45879460eda8/users/a909a5aa60b643e8a8029e651f262ad6/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:55.680981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.681333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.681695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.738687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 862/1728] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:55 2022] DELETE /identity/v3/users/d58840e417b641409e00df0771215b75 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:55.741350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:55.742273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] MFA Rules not processed for user `d8581d46699041d5a4dcb8a43034863e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:55.742799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.743751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.795152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:55.803355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:55.887537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 867/1729] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:55.891647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:55.896002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:55.899146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d009d5-01ca-47ee-8b3d-bb40fc0d7211 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:55.917675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:55.919612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:55.919913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:55.920786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] PATH_INFO: `/v3/projects/604127b144fe4b43be3f5a4a4739d51e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:55.924462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604127b144fe4b43be3f5a4a4739d51e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:55.928919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:55.999006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604127b144fe4b43be3f5a4a4739d51e', 'request_id': 'req-228d2889-cf67-44f1-8020-81a1728922a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.006645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604127b144fe4b43be3f5a4a4739d51e', 'request_id': 'req-228d2889-cf67-44f1-8020-81a1728922a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.017520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604127b144fe4b43be3f5a4a4739d51e', 'request_id': 'req-228d2889-cf67-44f1-8020-81a1728922a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.023082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70355f6-0679-459e-8178-2f23ebe4f099 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:56.031475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] The token cache is being invalidate because project 604127b144fe4b43be3f5a4a4739d51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:56.032123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.032631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-228d2889-cf67-44f1-8020-81a1728922a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.037351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 863/1730] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:55 2022] DELETE /identity/v3/projects/604127b144fe4b43be3f5a4a4739d51e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:56.038963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8f34-7600-4052-ab51-fa9070d87b6c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.039396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8f34-7600-4052-ab51-fa9070d87b6c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.039681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8f34-7600-4052-ab51-fa9070d87b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.077741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b1aa8f34-7600-4052-ab51-fa9070d87b6c None None] MFA Rules not processed for user `a909a5aa60b643e8a8029e651f262ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:56.079049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b1aa8f34-7600-4052-ab51-fa9070d87b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.100421 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:56.101052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.101737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.102018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.104222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.182042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 864/1731] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:56.184811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.185171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.185939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.186941 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:56.188297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5403c8-96f4-4de8-bbe1-0da05fc05f6f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:56.201057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 868/1732] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:56.203063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.203352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.203629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.216589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:56.218586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.227619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 865/1733] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:56.228653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.236676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.249870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] MFA Rules not processed for user `d8581d46699041d5a4dcb8a43034863e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:56.251105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.319400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:56.339058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:56.366910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 869/1734] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:56.368131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.372079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.404044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabcddc-f3c0-40fd-b6b1-809132945b71 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:56.413297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:56.413521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.413806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.414075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] PATH_INFO: `/v3/users/4e3c8b6fb2f646fd9da80371685c3281` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.415380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3c8b6fb2f646fd9da80371685c3281)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:56.416024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfe1eb-47fb-44bb-8aa5-5c6d96211b05 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:56.418256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:56.428431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:56.429480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.429859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.430206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.432376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.458490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:56.461623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:56.475486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 866/1735] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:56.476857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.479123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.513275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3c8b6fb2f646fd9da80371685c3281'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.533187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3c8b6fb2f646fd9da80371685c3281'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.534221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3c8b6fb2f646fd9da80371685c3281'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.535180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3c8b6fb2f646fd9da80371685c3281'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.547470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3c8b6fb2f646fd9da80371685c3281'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:56.578135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 870/1736] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:56 2022] DELETE /identity/v3/users/4e3c8b6fb2f646fd9da80371685c3281 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:56.579230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.580716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.628729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8571a-e3a0-41a2-966e-e2c567e42171 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:56.709014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c22a3-7e74-4a9f-b9bb-9b2985cff49a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:56.719851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:56.720479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.720753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.722365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.727942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.759617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:56.760361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:56.760512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:56.760788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:56.768079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.879418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:56.882239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:56.912573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 871/1737] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:56.913438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.918895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:56.926463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:56.940027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:56.942310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:56.968767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 867/1738] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:56.970463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:56.975220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.012983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6e02d-2630-4bb2-bbdb-d78cb0951a08 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.026452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.027303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.028540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.028970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] PATH_INFO: `/v3/users/d26716afd68a4fcfbbc323be98ea43d7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.031782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26716afd68a4fcfbbc323be98ea43d7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.034070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:57.109359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:57.147443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26716afd68a4fcfbbc323be98ea43d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.151053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70806e-f699-43c3-9e09-6403b5748ea7 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.157830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26716afd68a4fcfbbc323be98ea43d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.158876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26716afd68a4fcfbbc323be98ea43d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.159686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26716afd68a4fcfbbc323be98ea43d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.169798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.171451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.171931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.172415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.173472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26716afd68a4fcfbbc323be98ea43d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.182435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.235061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 868/1739] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:56 2022] DELETE /identity/v3/users/d26716afd68a4fcfbbc323be98ea43d7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:57.236381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:57.239193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.389668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef724-913b-4bd6-be80-bef745c2b9d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.394949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.395686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.396078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.396431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] PATH_INFO: `/v3/projects/1a1f2b35a3df4b91930a2add2ca240fc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.396769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.397768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1f2b35a3df4b91930a2add2ca240fc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.398686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:57.400059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:57.434235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 872/1740] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:57.435307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:57.442536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.474055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1f2b35a3df4b91930a2add2ca240fc', 'request_id': 'req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.479252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7319ac-0314-46de-935f-acd9893e3862 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.494618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1f2b35a3df4b91930a2add2ca240fc', 'request_id': 'req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.514793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1f2b35a3df4b91930a2add2ca240fc', 'request_id': 'req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.544301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] The token cache is being invalidate because project 1a1f2b35a3df4b91930a2add2ca240fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:57.544582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.546214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.552915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 869/1741] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:57 2022] DELETE /identity/v3/projects/1a1f2b35a3df4b91930a2add2ca240fc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:57.554067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:57.555705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.613594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:57.639580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.651788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.652119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.652443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] PATH_INFO: `/v3/projects/6a382126240d4d15a681c1c62a263175` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.653802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a382126240d4d15a681c1c62a263175)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.658424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:57.693058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628d5b2-7072-4006-a3b1-8b8e09bd42f3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.700701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.702367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.702537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.702760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.709603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.733244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:57.750015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a382126240d4d15a681c1c62a263175', 'request_id': 'req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.764796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a382126240d4d15a681c1c62a263175', 'request_id': 'req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.781312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a382126240d4d15a681c1c62a263175', 'request_id': 'req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.796952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] The token cache is being invalidate because project 6a382126240d4d15a681c1c62a263175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:57.797419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.798236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84d2cec0-41b9-49a9-87f5-fa734ea9d15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:57.800074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 873/1742] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:57 2022] DELETE /identity/v3/projects/6a382126240d4d15a681c1c62a263175 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:57.804008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.804370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.804719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.810163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.812472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:57.827598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 870/1743] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:57.828668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:57.831143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.850395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ec4a-eb73-44d5-9e10-51b619895110 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:57.853474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:57.854740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:57.930774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:57.931443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:57.931890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:57.932178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] PATH_INFO: `/v3/users/9675b9a1e0da4d31b7fbe96b0abb1801` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:57.934257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9675b9a1e0da4d31b7fbe96b0abb1801)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:57.937866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.005155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 874/1744] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:58.007101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.008605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.031452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c4f9-4ce9-4714-899c-36949e330149 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.033933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9675b9a1e0da4d31b7fbe96b0abb1801'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.039863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.040801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.041313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.041689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] PATH_INFO: `/v3/projects/751040822a3d462a8443274b80a92e40` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.043320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=751040822a3d462a8443274b80a92e40)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.046088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.054900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9675b9a1e0da4d31b7fbe96b0abb1801'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.056342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9675b9a1e0da4d31b7fbe96b0abb1801'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.062673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9675b9a1e0da4d31b7fbe96b0abb1801'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.076949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c6a26205-bd4e-4fb2-ae7d-c92f919b9001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9675b9a1e0da4d31b7fbe96b0abb1801'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.111416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 871/1745] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:57 2022] DELETE /identity/v3/users/9675b9a1e0da4d31b7fbe96b0abb1801 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:58.114155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.114548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.114920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.150291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:58.160764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751040822a3d462a8443274b80a92e40', 'request_id': 'req-7858050a-70f6-4afe-abb3-65067aacc898'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.171711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:58.176983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.179823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751040822a3d462a8443274b80a92e40', 'request_id': 'req-7858050a-70f6-4afe-abb3-65067aacc898'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.188918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751040822a3d462a8443274b80a92e40', 'request_id': 'req-7858050a-70f6-4afe-abb3-65067aacc898'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.219831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] The token cache is being invalidate because project 751040822a3d462a8443274b80a92e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:58.220187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.223284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.227339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 875/1746] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:58 2022] DELETE /identity/v3/projects/751040822a3d462a8443274b80a92e40 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:58.230684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.234833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.383110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 872/1747] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:58.384321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.387819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.402291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858050a-70f6-4afe-abb3-65067aacc898 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.420591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe90f9-350c-414f-8e4d-227a69799428 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.428704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.429345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:58.429517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.429675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.429907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] PATH_INFO: `/v3/users/090e239fc58949a3ac48e0eff6aafb8b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.433925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] RBAC: Authorizing `identity:delete_user(user_id=090e239fc58949a3ac48e0eff6aafb8b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.436085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.538876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090e239fc58949a3ac48e0eff6aafb8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.550344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.550568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090e239fc58949a3ac48e0eff6aafb8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.551062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.551166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090e239fc58949a3ac48e0eff6aafb8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.551357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.551727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090e239fc58949a3ac48e0eff6aafb8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.553110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.558362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.569228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8fdc0991-aee4-4407-8cce-a760d256558b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090e239fc58949a3ac48e0eff6aafb8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.599101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 873/1748] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:58 2022] DELETE /identity/v3/users/090e239fc58949a3ac48e0eff6aafb8b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:58.603104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.603104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.603304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.611502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.613237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.627744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 876/1749] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:58.652818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:58.656918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.781807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 874/1750] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:58.786163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.801357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1156e4c-4def-4258-8ce8-77a83bb49851 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.813066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.814843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.815551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.816062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.817075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:58.819061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.822237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7b7e0-6df3-455f-aea8-cf4c60c2d07a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.865320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 877/1751] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:58.866921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.885349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1315-4046-4b03-bc5e-c592db460e1a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.904944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.905869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.907956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.908401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.912812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.915208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.946169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.947032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.947506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.973149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] MFA Rules not processed for user `d8581d46699041d5a4dcb8a43034863e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:58.973824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:58.976157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.015162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 875/1752] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:58 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:59.016343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.155768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 878/1753] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:59.161297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e978c-4f0a-4030-aa79-27a0b9783b38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.168484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.169163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.169448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.171055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.172244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.175516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.183886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.186400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.206763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e108a9-6f02-44ae-a765-7a73c5bc6256 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.212536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.213795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.213795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.214000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] PATH_INFO: `/v3/projects/b4afdf975268425588e7d7f4058436e5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.215738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4afdf975268425588e7d7f4058436e5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.217835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.234987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 876/1754] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:59.237313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.259733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd9e2-69a0-4af3-8caf-78cef78798c6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.272420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.272985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.273149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.273421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.277385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.279123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.303393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4afdf975268425588e7d7f4058436e5', 'request_id': 'req-c356dfe9-ba5d-41f9-a369-50db77a67f23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.318602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4afdf975268425588e7d7f4058436e5', 'request_id': 'req-c356dfe9-ba5d-41f9-a369-50db77a67f23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.337594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 877/1755] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:59.339561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4afdf975268425588e7d7f4058436e5', 'request_id': 'req-c356dfe9-ba5d-41f9-a369-50db77a67f23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.343764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.385265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c779-03fb-4321-8d7a-5765d15ec2c7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.387221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] The token cache is being invalidate because project b4afdf975268425588e7d7f4058436e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:59.387524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.389570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.393720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 879/1756] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/projects/b4afdf975268425588e7d7f4058436e5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:59.395072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.398035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.488076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.533080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356dfe9-ba5d-41f9-a369-50db77a67f23 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.538203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.538971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.539257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.539547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] PATH_INFO: `/v3/projects/a5367e5b8d6d45739becaaf576bb346c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.541795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5367e5b8d6d45739becaaf576bb346c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.543888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.546577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.547211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.551042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.551779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.556620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.558287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60616bda-c92d-45b2-948e-39ae38bd9091 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.610788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 878/1757] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:59.613749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.614766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.616870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.630542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5367e5b8d6d45739becaaf576bb346c', 'request_id': 'req-5009ed8e-1694-4bc9-a11d-5ab70e31a274'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.643974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5367e5b8d6d45739becaaf576bb346c', 'request_id': 'req-5009ed8e-1694-4bc9-a11d-5ab70e31a274'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.656543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5367e5b8d6d45739becaaf576bb346c', 'request_id': 'req-5009ed8e-1694-4bc9-a11d-5ab70e31a274'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.677075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] The token cache is being invalidate because project a5367e5b8d6d45739becaaf576bb346c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:59.677870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.678102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.679270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.679414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.684763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 880/1758] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/projects/a5367e5b8d6d45739becaaf576bb346c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.686078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.689081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.794620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 879/1759] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:59.794620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.797064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.798850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009ed8e-1694-4bc9-a11d-5ab70e31a274 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.804180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.805351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.806330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.806961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] PATH_INFO: `/v3/projects/3f7d8432bdc442a1bc9357ceb9a96c60` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.808735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7d8432bdc442a1bc9357ceb9a96c60)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.811449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.829194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9742af-c619-40cd-94d9-7affa219503d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.842640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.846547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.848353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.849495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] PATH_INFO: `/v3/projects/50604b83540046219a9ec969664f4767/users/679d4c1ba4d44e2f9435f9a318f302cd/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.867050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.875567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=50604b83540046219a9ec969664f4767, user_id=679d4c1ba4d44e2f9435f9a318f302cd, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.879375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff487fd-241e-4c38-8ca5-969fbb1aa35c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.922470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 880/1760] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:59 2022] PUT /identity/v3/projects/50604b83540046219a9ec969664f4767/users/679d4c1ba4d44e2f9435f9a318f302cd/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.922470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4f16e-b353-4f82-8516-c197a0719191 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.922470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4f16e-b353-4f82-8516-c197a0719191 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.922470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4f16e-b353-4f82-8516-c197a0719191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.922900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7d8432bdc442a1bc9357ceb9a96c60', 'request_id': 'req-a121fea7-b923-4b36-8126-5a21c96a1870'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.930382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7d8432bdc442a1bc9357ceb9a96c60', 'request_id': 'req-a121fea7-b923-4b36-8126-5a21c96a1870'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.934467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.939348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-88b4f16e-b353-4f82-8516-c197a0719191 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.940658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88b4f16e-b353-4f82-8516-c197a0719191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.942059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7d8432bdc442a1bc9357ceb9a96c60', 'request_id': 'req-a121fea7-b923-4b36-8126-5a21c96a1870'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.959725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] The token cache is being invalidate because project 3f7d8432bdc442a1bc9357ceb9a96c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:59.960077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.960852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a121fea7-b923-4b36-8126-5a21c96a1870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.966022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 881/1761] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/projects/3f7d8432bdc442a1bc9357ceb9a96c60 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.967972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4076ed-e203-4531-9e3b-2c681346de22 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.968359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4076ed-e203-4531-9e3b-2c681346de22 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.968510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4076ed-e203-4531-9e3b-2c681346de22 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.970400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 882/1762] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 15:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:59.971486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4c05-1426-4059-bdf2-7182b76b54b8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.971672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4c05-1426-4059-bdf2-7182b76b54b8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.971918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4c05-1426-4059-bdf2-7182b76b54b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.992987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d47b4c05-1426-4059-bdf2-7182b76b54b8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.994218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d47b4c05-1426-4059-bdf2-7182b76b54b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.033494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 881/1763] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.035230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dc41-b501-4b14-91bc-24a33369e2af None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.035671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dc41-b501-4b14-91bc-24a33369e2af None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.036083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dc41-b501-4b14-91bc-24a33369e2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.065934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5b75dc41-b501-4b14-91bc-24a33369e2af None None] MFA Rules not processed for user `679d4c1ba4d44e2f9435f9a318f302cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.067167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b75dc41-b501-4b14-91bc-24a33369e2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.071953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 883/1764] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.073516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017b4a-b18e-49f8-9a71-90b6ca7bdb11 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.073637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017b4a-b18e-49f8-9a71-90b6ca7bdb11 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.073842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017b4a-b18e-49f8-9a71-90b6ca7bdb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.094897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-57017b4a-b18e-49f8-9a71-90b6ca7bdb11 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.096116 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-57017b4a-b18e-49f8-9a71-90b6ca7bdb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.104431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 884/1765] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.106819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381354-4f92-4281-8463-b2051df0a06b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.108932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381354-4f92-4281-8463-b2051df0a06b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.110551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381354-4f92-4281-8463-b2051df0a06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.138103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3e381354-4f92-4281-8463-b2051df0a06b None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.139953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3e381354-4f92-4281-8463-b2051df0a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.150074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 882/1766] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.152694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.153058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.153890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.173921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.175226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.183690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 883/1767] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.184531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.199838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788381-f58c-4e2f-ac29-9b9d5196c89e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.205781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.206571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.206849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.207120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.207900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:00.208207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.209866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.209973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 885/1768] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.211286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.211697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.211986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.226604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 884/1769] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:00.228968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.230885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.233651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.234871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.243969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 886/1770] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:00.247544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.251110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10a3eb-ce27-48a1-8670-b4fbe173b48e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.257679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.259149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.259599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.260012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.261446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.264462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.269856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348fa8-6453-4961-8167-e2d8536d363c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.275487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.276827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.276827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.276827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.277563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:00.277939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.280656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.296620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 887/1771] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:00.297475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.302534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.303945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 885/1772] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.305411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.321052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed8e8c-b839-4e65-9fd1-d9561595c945 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.358401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.385384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5781f1-7640-4a0a-84d8-e10eaa7f693a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.392315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.392933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.393212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.393487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.394134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.396176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.396948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.398562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.398979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.399359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.415539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.415539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.421007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 888/1773] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:00.422030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.438738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 886/1774] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.439562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.454292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bf58-5eec-4e86-a61d-78d74bf979dd service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.475610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.538768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6616f-ca7b-45c6-972a-7c0838653f1c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.549288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.550057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.550344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.550674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.551321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.554448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.563387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.564095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.564496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.564904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.565625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:00.566058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.568169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.589993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 889/1775] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:00.590726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.607427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617c9ed-5737-4737-9da9-b6133202107e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.611907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 887/1776] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.613449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.699824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.700667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.701100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.701896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6d7f-6135-4c2c-bc9f-e11615ee9526 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.703254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.704250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.707072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.711411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.712210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.712706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.713233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.715569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.719454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.749485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 890/1777] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.750495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.755019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 888/1778] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.755019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.770704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ab1d-ce68-4d43-b271-1d6de2f36e20 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.776479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.777228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.777610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.777975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.778795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.781525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.788652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4ab78-4be1-4614-ab74-56f601716a92 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.799173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.800139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.801340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.801631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.806794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.809475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 889/1779] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/users => generated 283 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.810558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.814828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.828317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226659b6-8689-4936-af02-6c80266554a4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.835152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.836328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.836749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.837307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.838279 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.840646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.854197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 890/1780] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:00.855288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.867647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb457f-52b1-4bd9-82ff-e156623160de admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.875807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.876586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.876973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.877352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.877521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 891/1781] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.881824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.882042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.882203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.882203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.898454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 891/1782] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:00.898454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.909795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04973b-23c6-4605-bee4-28d8f223ab85 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.988824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.989650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.990206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.990648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] PATH_INFO: `/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d/users/eff542f7e5dc4be9801d6994a268420f/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.018397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8b92ac2ba44579b4ebb3ee889b032d, user_id=eff542f7e5dc4be9801d6994a268420f, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.021508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.060295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 892/1783] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:00 2022] PUT /identity/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d/users/eff542f7e5dc4be9801d6994a268420f/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.062218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258068-c5ba-4a22-a07d-c88856b5abcc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.064594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.116985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.181540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.183000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.183692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.184107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.185105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.189459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.207298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 892/1784] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.207298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.208693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a7495-2f21-4634-9893-572705bc1dcc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.218504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.218789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5beb-3829-4fdd-aed0-c7a972c5421e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.219388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.219798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.220175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.221864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.223693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.250617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.251248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.251531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.251807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.252476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.254131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.257469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 893/1785] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:01.264443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.286263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab6285-8ded-4e47-beca-be0c82d17dd2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.292584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.293356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.293747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.294126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.295559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.296531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 893/1786] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.297446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.297675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.333959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ec90-d22d-4e16-a648-448f890d62e4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.341007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 894/1787] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.342942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.355214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.355866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.356132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.356477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.357044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.358854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.364530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40fea9-0496-4b67-ab39-37ff61266738 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.378948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 894/1788] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.380584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.380959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.381609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.381911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.382771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.383431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.385107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.393792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274ee1-53a5-45d8-b4cd-d6a3a194be19 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.402444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 895/1789] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.402444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.412352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.413028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.413306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.413578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] PATH_INFO: `/v3/projects/fb33cd0774ee4ae8992645c1be7554ff/users/b638d111d9c34b4c80f4370aea97eeff/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.417193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a3757-f578-457e-9884-2ad42b8de224 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.422633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.423316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.423704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.423979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] PATH_INFO: `/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d/users/eff542f7e5dc4be9801d6994a268420f/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.425978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8b92ac2ba44579b4ebb3ee889b032d, user_id=eff542f7e5dc4be9801d6994a268420f, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.429371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.456510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 896/1790] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d/users/eff542f7e5dc4be9801d6994a268420f/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.457686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.476974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb33cd0774ee4ae8992645c1be7554ff, user_id=b638d111d9c34b4c80f4370aea97eeff, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.479958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.505300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 895/1791] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/fb33cd0774ee4ae8992645c1be7554ff/users/b638d111d9c34b4c80f4370aea97eeff/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.507994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.569550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.613430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd241-a76b-46a9-b413-163758fa6c11 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.619282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.620670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.620965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.621359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.621980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.624392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab32ff-b63c-443e-9e05-b38bbfd981a8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.636744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240629-0b0a-47a9-8ed3-ffad580bb9db admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.642990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 896/1792] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.643632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.644434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.644764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.645109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] PATH_INFO: `/v3/projects/c7b777f33a1d42ab9028604e0d01463a/users/d493cd87124a40e5b30fae49ab597796/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.648026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.648315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.648589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.664398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b777f33a1d42ab9028604e0d01463a, user_id=d493cd87124a40e5b30fae49ab597796, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.667201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13beba-1a9d-4bdd-b60e-fe5398cfdc9e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.687127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] MFA Rules not processed for user `eff542f7e5dc4be9801d6994a268420f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.689548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.691901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 897/1793] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/c7b777f33a1d42ab9028604e0d01463a/users/d493cd87124a40e5b30fae49ab597796/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.693631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.694009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.694395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.734688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] MFA Rules not processed for user `b638d111d9c34b4c80f4370aea97eeff`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.735905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.814440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.848760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 897/1794] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:01.851878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.854518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.906717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 898/1795] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:01.907962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.993438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ee658-4136-4e35-b4b2-1b3ac0304775 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.997539 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.998294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.998609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.998890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.000726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.061241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51267246-a063-47ec-b609-b70bf01a4cd4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.073765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.074956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.075273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.075441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] PATH_INFO: `/v3/projects/64077172128249a1bf0d8b58da80a450/users/532e66482b9d4ed9a33dff51fcf37421/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.096826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=64077172128249a1bf0d8b58da80a450, user_id=532e66482b9d4ed9a33dff51fcf37421, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.100912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.103476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.105404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee81c91-bd6f-4923-a831-34d5d0c5e359 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.139233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 898/1796] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:02.142844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cfd5-2258-4337-9291-03c46aa83489 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.143246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cfd5-2258-4337-9291-03c46aa83489 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.144634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cfd5-2258-4337-9291-03c46aa83489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.150990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 899/1797] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/64077172128249a1bf0d8b58da80a450/users/532e66482b9d4ed9a33dff51fcf37421/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:02.156483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.163974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.245944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-67c4cfd5-2258-4337-9291-03c46aa83489 None None] MFA Rules not processed for user `d493cd87124a40e5b30fae49ab597796`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:02.249751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-67c4cfd5-2258-4337-9291-03c46aa83489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.254762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:02.323099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e387cd5-cf9f-4d2f-941c-744178dd3eec admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.334046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.334932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.336873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.337536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.455524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 899/1798] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:02.457640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.457939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.458217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.471411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.473108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.485447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 900/1799] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:02.497208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.508195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] MFA Rules not processed for user `532e66482b9d4ed9a33dff51fcf37421`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:02.508681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6b001-12cf-4853-ae97-6ba6b58db0eb service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.509883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.512943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.515161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.515564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.516112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.548164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:02.674468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 900/1800] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:02.674468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.728334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.729969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25208b-325b-46e6-bb42-7a239f387ea3 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.752084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 901/1801] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:02.858985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520e7c0-8a52-40be-abbe-68c0d27a6fdc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.875250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.875898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.876179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.876499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.877320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.879503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.911273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.911273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.911494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.956782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 901/1802] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:02 2022] POST /identity/v3/projects => generated 361 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:02.957643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.074327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:03.077172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:03.087047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0257da-9e23-4621-931d-36d3764a9972 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.095884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.097930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.098536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.099657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.190940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:03.237850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 902/1803] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:03.239044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.291624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.294325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.309409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 902/1804] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:03.421848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e2dac-3bad-487f-a933-02f23870e64b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.432636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.433365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.433951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.434368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.435102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.438723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.465260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.466929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:03.476362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:03.503407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 903/1805] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:03 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:03.504282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.550504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7397-6b05-4e69-a65f-c79afcd75a08 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.568409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.569023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.569307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.569579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.570237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.571895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.623429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 904/1806] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:03.624598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.672873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96547b8-8430-461a-a239-1575183b35b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.693060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.693545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b8fd-7eeb-4254-b50c-3c551dac2f5a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.693745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.694031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.694308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] PATH_INFO: `/v3/projects/63f912b9b83147598bc990a2e5f55957/users/c6f116407d79465fb4169efb32c025b4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.704990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.705902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.706382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.706882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.728072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.729900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.751114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 903/1807] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:43:03.752089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f912b9b83147598bc990a2e5f55957, user_id=c6f116407d79465fb4169efb32c025b4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.762330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.845668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 905/1808] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:03 2022] PUT /identity/v3/projects/63f912b9b83147598bc990a2e5f55957/users/c6f116407d79465fb4169efb32c025b4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:03.850101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.866267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaa6d-e021-48b6-ac3b-7aa7b46f126a service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.916378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.098515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.099150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.099432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.100686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.103131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.107069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.134175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 904/1809] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:04.149005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.181459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc69d8-92c0-40a8-b986-252ba7cce495 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.185134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ab4b7-6020-456b-8715-df05ea617972 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.189721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.192111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.192432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.192765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.197554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.201562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.201897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.202185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] PATH_INFO: `/v3/projects/63f912b9b83147598bc990a2e5f55957/users/c6f116407d79465fb4169efb32c025b4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.209547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f912b9b83147598bc990a2e5f55957, user_id=c6f116407d79465fb4169efb32c025b4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.219464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0a3d2-9653-46b6-a182-d8d25747d475 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.238759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 905/1810] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:04 2022] PUT /identity/v3/projects/63f912b9b83147598bc990a2e5f55957/users/c6f116407d79465fb4169efb32c025b4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:04.253351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.253759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.254846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.393333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] MFA Rules not processed for user `c6f116407d79465fb4169efb32c025b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:04.399114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.400632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eda491-53f6-44b3-9a9a-5e2086974fa6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.402000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.405554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:04.420687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 906/1811] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:04.719335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:04.785812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 906/1812] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 530 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:04.985001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.986291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.986748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.017040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:05.018511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:05.055012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:05.171798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2583d-8552-4845-8ed3-c1484c103992 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:05.179426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:05.180906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:05.181714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:05.182128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.197139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 907/1813] 10.222.0.75 () {58 vars in 1015 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:05.202528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:05.221051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:05.221950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:05.223110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:05.251718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7da3-a9f7-4671-b353-a35a3f71be1c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:05.261338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 907/1814] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:05.264713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:05.269152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:05.269588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:05.270268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.293210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:05.297871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9878a-0e1e-421f-93a6-9f4a8d5cf2f6 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:05.319359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 908/1815] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:05.985841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:05.999025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56824480-e44b-4c26-b638-4c60308d82a0 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.005678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.006537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.007934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.008474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.256050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.257055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425bae9-fad2-4333-bc1b-80caa29eada0 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.292255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 908/1816] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:06.918087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.919284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.920372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.921885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 909/1817] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:43:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:43:06.936091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.937316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.937810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.965395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.967267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.097683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 909/1818] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:07.110699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.112683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.134480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633e66-2689-4ea9-a712-626adb0fad67 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.141579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.142193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.142503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.142778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.154830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.156255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53872e3-4e0b-44ec-a404-91ba088ed03e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.167396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 910/1819] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:17.081719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:17.162715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=36 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:17.976204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:18.041799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d9683-af0d-4e78-a8a9-70addf29c322 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:18.047446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:18.048076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:18.048360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:18.048634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] PATH_INFO: `/v3/users/d493cd87124a40e5b30fae49ab597796` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:18.049789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d493cd87124a40e5b30fae49ab597796)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:18.052476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:18.102041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd493cd87124a40e5b30fae49ab597796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:18.109812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd493cd87124a40e5b30fae49ab597796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:18.110530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd493cd87124a40e5b30fae49ab597796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:18.111146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd493cd87124a40e5b30fae49ab597796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:18.119806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd493cd87124a40e5b30fae49ab597796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:18.142282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 910/1820] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:17 2022] DELETE /identity/v3/users/d493cd87124a40e5b30fae49ab597796 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:18.146959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:18.147474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:18.147821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:18.180585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:18.181980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:18.332027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 911/1821] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:18.339195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:18.341263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:18.514964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246564-0384-4118-9717-2d1e42dbcbc3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:18.520062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:18.520226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:18.520442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:18.520724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:18.549518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:18.551987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:18.595501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 911/1822] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:19.037051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.038706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.053295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf2056-2085-4bdd-bd2c-3ca2abe96c64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.058297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.059308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.059579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.059870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] PATH_INFO: `/v3/projects/c7b777f33a1d42ab9028604e0d01463a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.061758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b777f33a1d42ab9028604e0d01463a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.063991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.153047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b777f33a1d42ab9028604e0d01463a', 'request_id': 'req-dbe021d8-11ff-4562-be88-7b8dc7addfdd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.166512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b777f33a1d42ab9028604e0d01463a', 'request_id': 'req-dbe021d8-11ff-4562-be88-7b8dc7addfdd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.179014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b777f33a1d42ab9028604e0d01463a', 'request_id': 'req-dbe021d8-11ff-4562-be88-7b8dc7addfdd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.193839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] The token cache is being invalidate because project c7b777f33a1d42ab9028604e0d01463a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:19.194818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.194818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.196608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 912/1823] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/projects/c7b777f33a1d42ab9028604e0d01463a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:19.200644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.213085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6980b-5b5c-489b-a426-4f607afe9419 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.234603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:19.294638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.295398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.295699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.295976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] PATH_INFO: `/v3/users/c6f116407d79465fb4169efb32c025b4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.297102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f116407d79465fb4169efb32c025b4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.299081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.347896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f116407d79465fb4169efb32c025b4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.355561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f116407d79465fb4169efb32c025b4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.356578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f116407d79465fb4169efb32c025b4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.357252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f116407d79465fb4169efb32c025b4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.364468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f116407d79465fb4169efb32c025b4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.381422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 912/1824] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/users/c6f116407d79465fb4169efb32c025b4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:19.630618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.632143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.653772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe021d8-11ff-4562-be88-7b8dc7addfdd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.725888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.728530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.768140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.768813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.769172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.769453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] PATH_INFO: `/v3/users/36abb8abc44e438398b8927f70016a86` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.771246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=36abb8abc44e438398b8927f70016a86)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.773301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.807035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:19.834326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36abb8abc44e438398b8927f70016a86'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.842872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36abb8abc44e438398b8927f70016a86'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.843049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400623-95fb-4697-a736-e3a64e962a60 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.843667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36abb8abc44e438398b8927f70016a86'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.844322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36abb8abc44e438398b8927f70016a86'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.848356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.849059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.849310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.849678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] PATH_INFO: `/v3/projects/63f912b9b83147598bc990a2e5f55957` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.852088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f912b9b83147598bc990a2e5f55957)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.853797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26b63fdd-3b04-4a65-acf9-c3962b7e20ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36abb8abc44e438398b8927f70016a86'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.855588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.877865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 913/1825] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/users/36abb8abc44e438398b8927f70016a86 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:19.882370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.882682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.882951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.908442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:19.913219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f912b9b83147598bc990a2e5f55957', 'request_id': 'req-eceed42c-9222-43fb-9117-d645e604c242'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.914354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:19.915604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.924193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f912b9b83147598bc990a2e5f55957', 'request_id': 'req-eceed42c-9222-43fb-9117-d645e604c242'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.931462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f912b9b83147598bc990a2e5f55957', 'request_id': 'req-eceed42c-9222-43fb-9117-d645e604c242'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.943293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] The token cache is being invalidate because project 63f912b9b83147598bc990a2e5f55957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:19.943587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.944241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eceed42c-9222-43fb-9117-d645e604c242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.945759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 913/1826] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/projects/63f912b9b83147598bc990a2e5f55957 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:19.951130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a25d6-45b4-4077-bf24-6d1439b410ac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.951946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a25d6-45b4-4077-bf24-6d1439b410ac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.952218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a25d6-45b4-4077-bf24-6d1439b410ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.973615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-456a25d6-45b4-4077-bf24-6d1439b410ac None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:19.974970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-456a25d6-45b4-4077-bf24-6d1439b410ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.005946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 914/1827] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:20.017410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.019743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.044664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 914/1828] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:20.054225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.054540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.054812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.076137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:20.077407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.087445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 915/1829] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:20.092736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.108804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770a64-bee6-4423-9d03-c9cb6d789fb4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.110711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.111369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.111664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.111900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.112851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:20.115064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eece2a1-e100-4e9a-9cd0-d3640472ffc7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.115452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.117227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.121371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.122355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.122824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.123839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.132691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 916/1830] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:20.137243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.138884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.140871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.148259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089464-c7f4-4e8b-a569-ae17f3721a56 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.153582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 915/1831] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:20.163488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.164403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.164689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.164965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.165611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.167613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.204331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 917/1832] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:20.208795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.218686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8820e-3ff1-4d45-84f9-d4874e84cf9d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.274786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:20.295801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.296480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.296816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.297143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.297790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.299929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.347997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 916/1833] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:20.349297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.365863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a56a-ad83-45f7-8737-abc744264827 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.371462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.372186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.372486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.372963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.374232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.376215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.392960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.393898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 918/1834] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:20.427659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6790f7-bbaf-4aec-acbd-7603e9b0bae0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.432688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.433312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.433847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.434122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.434802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.439539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.463393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 917/1835] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:20.469405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.487686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b06f-7623-40bb-be13-4dc1310120af admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.494498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.495195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.495479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.495755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] PATH_INFO: `/v3/projects/7d0dddf93c164b54b4ba6a0522569d8c/users/58060a3e0d2840dd8d73d1e9fb5c553a/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.511032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.512848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.518905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0dddf93c164b54b4ba6a0522569d8c, user_id=58060a3e0d2840dd8d73d1e9fb5c553a, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.522195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9a68-a3a4-4d52-8ef1-2e54e680341c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.535672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb1b4-6a42-4141-a2a0-6cd08c0cc60b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.572080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 919/1836] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:20 2022] PUT /identity/v3/projects/7d0dddf93c164b54b4ba6a0522569d8c/users/58060a3e0d2840dd8d73d1e9fb5c553a/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:20.582820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.583111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.583389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.601048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.601490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.601761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.602036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] PATH_INFO: `/v3/projects/b20b0ef06a1842bc890551a27f696999` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.603198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20b0ef06a1842bc890551a27f696999)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.605081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.612701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] MFA Rules not processed for user `58060a3e0d2840dd8d73d1e9fb5c553a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:20.614006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.653489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20b0ef06a1842bc890551a27f696999', 'request_id': 'req-258ca1f6-f57a-4413-b8c7-00b8b4dac627'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.660293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20b0ef06a1842bc890551a27f696999', 'request_id': 'req-258ca1f6-f57a-4413-b8c7-00b8b4dac627'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.660426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:20.669224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20b0ef06a1842bc890551a27f696999', 'request_id': 'req-258ca1f6-f57a-4413-b8c7-00b8b4dac627'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.682394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] The token cache is being invalidate because project b20b0ef06a1842bc890551a27f696999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:20.682709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.683606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.684990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 920/1837] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:20.685305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 918/1838] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:20 2022] DELETE /identity/v3/projects/b20b0ef06a1842bc890551a27f696999 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:20.689140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.689373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.690727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.699793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325a528-525c-4e50-a0e0-406a2f9b099c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.751088 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:20.756987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ca1f6-f57a-4413-b8c7-00b8b4dac627 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.760663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.761682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.761990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.762264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] PATH_INFO: `/v3/users/a909a5aa60b643e8a8029e651f262ad6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.764188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.764663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a909a5aa60b643e8a8029e651f262ad6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.764872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.765092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.765742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.766630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.766732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.768929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.799855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 919/1839] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:20.804104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.819958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a909a5aa60b643e8a8029e651f262ad6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.828291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a909a5aa60b643e8a8029e651f262ad6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.829005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a909a5aa60b643e8a8029e651f262ad6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.829754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a909a5aa60b643e8a8029e651f262ad6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.837618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a909a5aa60b643e8a8029e651f262ad6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.860544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 921/1840] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:20 2022] DELETE /identity/v3/users/a909a5aa60b643e8a8029e651f262ad6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:20.876364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789090b-1554-423f-8986-9da9d049b90c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.962552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.963374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.963784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.964106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.964793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.966806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.992960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 920/1841] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:20.996947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.999119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.013302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489293f-b3a9-4a1a-ac16-b00e77dd1094 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.018632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.019667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.019667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.034197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.034197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.034197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.039666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 922/1842] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:21.040449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.051118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91cc4c-8975-4ae6-846a-16b2ee872436 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.056426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.057090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.057373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.057653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] PATH_INFO: `/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e/users/5e68df220e0341c4afc08ee6762bb2ea/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.075261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e0d0bcdac44c4ab5b5c139dc48554e, user_id=5e68df220e0341c4afc08ee6762bb2ea, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.078171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.092703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 921/1843] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:21 2022] PUT /identity/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e/users/5e68df220e0341c4afc08ee6762bb2ea/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:21.096810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.107567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3d389-1975-42f4-9fb5-a955bdb05c60 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.163460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.165552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.190829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.205328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.206028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.206879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.207452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.208328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.210018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.222819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 923/1844] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:21.226360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.231630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.237587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46412-6eb0-4103-bf19-33b2a310db13 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.242935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.243636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.243919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.244199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] PATH_INFO: `/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e/users/5e68df220e0341c4afc08ee6762bb2ea/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.246145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95280deb-b795-4970-8a48-c22d37b6f1dd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.247773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e0d0bcdac44c4ab5b5c139dc48554e, user_id=5e68df220e0341c4afc08ee6762bb2ea, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.251327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14de43-ede4-4242-8f8a-0f146f153fb2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.251602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.252294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.252575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.252878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] PATH_INFO: `/v3/projects/ca244458caa44504905f45879460eda8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.254000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca244458caa44504905f45879460eda8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.255923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.268693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 924/1845] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:21 2022] PUT /identity/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e/users/5e68df220e0341c4afc08ee6762bb2ea/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:21.278721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.279007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.280302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.306348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca244458caa44504905f45879460eda8', 'request_id': 'req-c9825276-1461-4a05-9421-896e58320396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.310485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] MFA Rules not processed for user `5e68df220e0341c4afc08ee6762bb2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:21.311768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.312165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca244458caa44504905f45879460eda8', 'request_id': 'req-c9825276-1461-4a05-9421-896e58320396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.320322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca244458caa44504905f45879460eda8', 'request_id': 'req-c9825276-1461-4a05-9421-896e58320396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.332194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] The token cache is being invalidate because project ca244458caa44504905f45879460eda8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:21.332498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.333139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9825276-1461-4a05-9421-896e58320396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.334880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 922/1846] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:21 2022] DELETE /identity/v3/projects/ca244458caa44504905f45879460eda8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:21.339438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27238-c337-454c-ad2d-360593f8d64d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.339748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27238-c337-454c-ad2d-360593f8d64d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.340043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27238-c337-454c-ad2d-360593f8d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.368213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-64a27238-c337-454c-ad2d-360593f8d64d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:21.369484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64a27238-c337-454c-ad2d-360593f8d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.397499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 925/1847] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:21.409231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.410752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.430661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 923/1848] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:21.439897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.440174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.440457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.461359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:21.462531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.470912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 924/1849] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:21.471639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9c693-2c0f-4bdf-a61d-e7e07330d9c4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.474810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.475735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.476352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.476641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.476914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.488308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.490430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.492787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06177b1b-3a96-4d90-ba37-c5a59b934cf7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.497847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.499123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.499529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.499814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.500564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:21.500796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.501696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 926/1850] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:21.503392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.516398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 925/1851] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:43:21.521137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.533917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746075ab-e242-46c1-907a-e2689d3a6ee5 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.538915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.539516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.539791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.540056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.540852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.542608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.576915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 927/1852] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:21.583152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.594601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df3e9-1e18-4f35-99bb-36d3d759bcca admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.670771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.671574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.671997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.672555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.673462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.675459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.690089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.707630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 926/1853] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:21.713352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.735896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7f1a-edd7-467b-ba4c-d7e0b7e6c9c1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.741538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.742164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.742479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.742761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.743436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.745082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.770538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 928/1854] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:21.770752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.780960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb4d1-4345-4e10-bb5d-54d0294f47d6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.786509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.787256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.787548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.787900 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.788616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.790347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391cfae-8ba6-4ef5-846c-f64b40ec7224 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.802232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 927/1855] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:21.806438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.817467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059626a8-ba73-4322-8f7c-6dd8756a86ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.822170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.822850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.823128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.823400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] PATH_INFO: `/v3/projects/625adb5fa2a54a029c1c5326340a1ccf/users/bd4f1605139f4a239f51dcb3c25156c8/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.840149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625adb5fa2a54a029c1c5326340a1ccf, user_id=bd4f1605139f4a239f51dcb3c25156c8, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.843714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.859715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 929/1856] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:21 2022] PUT /identity/v3/projects/625adb5fa2a54a029c1c5326340a1ccf/users/bd4f1605139f4a239f51dcb3c25156c8/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:21.870119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.870428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.870715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.902945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] MFA Rules not processed for user `bd4f1605139f4a239f51dcb3c25156c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:21.904809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.980257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 928/1857] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:21.985219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.994124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:22.059730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce9e4-a1b2-4368-8d7d-9266dee2c104 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.065399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.066102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.066488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.066846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.067629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.069630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.117081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 930/1858] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:22.121262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.138114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d006-995b-4ad8-811a-63c6129ccf85 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.255155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.255811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.256158 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.256500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.257265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.259138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.287920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 929/1859] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:22 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:22.292036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.302991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac2c58-9d6a-408b-9099-08f8f9ea329a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.307646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.308612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.308770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.309049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.309698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.311367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.323234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 931/1860] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:22.327943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.340456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efee7e9-6026-45a5-9fa3-8f810bc6c85f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.345800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.346520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.346788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.347068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] PATH_INFO: `/v3/projects/f7da70bbd8504cd697ef3cf97f968f68/users/d1d492e9d4974af682f7ffef161d80cb/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.362894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7da70bbd8504cd697ef3cf97f968f68, user_id=d1d492e9d4974af682f7ffef161d80cb, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.365760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.380093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 930/1861] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:22 2022] PUT /identity/v3/projects/f7da70bbd8504cd697ef3cf97f968f68/users/d1d492e9d4974af682f7ffef161d80cb/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:22.384025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.395266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3992-def4-45dd-90cd-aa6c8c94c6be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.469314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.470143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.470582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.471143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.475112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.475112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27cca3-fef6-4aad-a320-a3cf11f041b5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.486397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 932/1862] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:22.491551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.502917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133ac8-827b-4c8e-8067-eba0d7481008 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.507936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.508588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.508862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.509408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] PATH_INFO: `/v3/projects/f7da70bbd8504cd697ef3cf97f968f68/users/d1d492e9d4974af682f7ffef161d80cb/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.511384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7da70bbd8504cd697ef3cf97f968f68, user_id=d1d492e9d4974af682f7ffef161d80cb, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.514601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.527666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 931/1863] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:22 2022] PUT /identity/v3/projects/f7da70bbd8504cd697ef3cf97f968f68/users/d1d492e9d4974af682f7ffef161d80cb/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:22.539964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.540326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.540673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.573201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] MFA Rules not processed for user `d1d492e9d4974af682f7ffef161d80cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:22.574515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.648260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 933/1864] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:22.661040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.691947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:22.726037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509e4c-eb88-4fea-b386-6b5337a7d1bf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.729493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.730166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.730467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.730743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.740927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.742273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.751999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 932/1865] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:22.841965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.847070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:22.853936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ada00-16b2-4cd8-a873-6213aa257f62 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.861025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.862371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.862687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.862962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.930704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.932443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c238d-7c42-4312-adaa-b6529822d3cf service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.947398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 934/1866] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:23.101806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:23.103293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:23.176170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439482-d4b4-4b29-ae89-77bc1aa9fda7 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:23.181380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:23.182015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:23.182398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:23.182619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] PATH_INFO: `/v3/users/58060a3e0d2840dd8d73d1e9fb5c553a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:23.184166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=58060a3e0d2840dd8d73d1e9fb5c553a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:23.186971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:23.281343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58060a3e0d2840dd8d73d1e9fb5c553a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.294621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58060a3e0d2840dd8d73d1e9fb5c553a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.294621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58060a3e0d2840dd8d73d1e9fb5c553a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.294621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58060a3e0d2840dd8d73d1e9fb5c553a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.302742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58060a3e0d2840dd8d73d1e9fb5c553a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.330484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 933/1867] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:23 2022] DELETE /identity/v3/users/58060a3e0d2840dd8d73d1e9fb5c553a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:23.335797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:23.336271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:23.336375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:23.362126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:23.363409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:23.424583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 935/1868] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:23.434462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:23.436591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:23.505201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:23.511002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e40669-e247-4136-94c7-a8db0702f21d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:23.515838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:23.516456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:23.516917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:23.517199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:23.526448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:23.527917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:23.537277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 934/1869] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:23.809410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:23.811879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:23.826060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a07339-b90a-420c-9b34-a330260831f2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:23.832008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:23.833138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:23.833138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:23.833352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] PATH_INFO: `/v3/projects/7d0dddf93c164b54b4ba6a0522569d8c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:23.834355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d0dddf93c164b54b4ba6a0522569d8c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:23.836300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:23.894819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d0dddf93c164b54b4ba6a0522569d8c', 'request_id': 'req-76765da8-5a6c-43cf-94f9-388b535392e3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.903811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d0dddf93c164b54b4ba6a0522569d8c', 'request_id': 'req-76765da8-5a6c-43cf-94f9-388b535392e3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.911913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d0dddf93c164b54b4ba6a0522569d8c', 'request_id': 'req-76765da8-5a6c-43cf-94f9-388b535392e3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.923491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] The token cache is being invalidate because project 7d0dddf93c164b54b4ba6a0522569d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:23.923778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.924440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:23.926144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 936/1870] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:23 2022] DELETE /identity/v3/projects/7d0dddf93c164b54b4ba6a0522569d8c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:23.930035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:23.939828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd98-842b-4696-9747-2dd155075589 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:24.002010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:24.002822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:24.003197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:24.003566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] PATH_INFO: `/v3/users/5e68df220e0341c4afc08ee6762bb2ea` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:24.004813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e68df220e0341c4afc08ee6762bb2ea)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:24.007265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:24.059369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e68df220e0341c4afc08ee6762bb2ea'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.067091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e68df220e0341c4afc08ee6762bb2ea'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.068579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e68df220e0341c4afc08ee6762bb2ea'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.068579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e68df220e0341c4afc08ee6762bb2ea'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.076064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08200b4a-317c-4905-8ae1-dc5a9e416f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e68df220e0341c4afc08ee6762bb2ea'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.095372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 935/1871] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:23 2022] DELETE /identity/v3/users/5e68df220e0341c4afc08ee6762bb2ea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:24.703722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:24.705159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:24.720675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76765da8-5a6c-43cf-94f9-388b535392e3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:24.751128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:24.833853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:24.834645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:24.834935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:24.835306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] PATH_INFO: `/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:24.836987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e0d0bcdac44c4ab5b5c139dc48554e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:24.838677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:24.896975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e0d0bcdac44c4ab5b5c139dc48554e', 'request_id': 'req-9ee6607f-d1c1-4714-a14a-38e4569e6863'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.908814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e0d0bcdac44c4ab5b5c139dc48554e', 'request_id': 'req-9ee6607f-d1c1-4714-a14a-38e4569e6863'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.916818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e0d0bcdac44c4ab5b5c139dc48554e', 'request_id': 'req-9ee6607f-d1c1-4714-a14a-38e4569e6863'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.929032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] The token cache is being invalidate because project 67e0d0bcdac44c4ab5b5c139dc48554e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:24.929324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.930006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9ee6607f-d1c1-4714-a14a-38e4569e6863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:24.931962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 937/1872] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:24 2022] DELETE /identity/v3/projects/67e0d0bcdac44c4ab5b5c139dc48554e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:24.937597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:24.937597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:24.938188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:24.959306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:24.960586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:25.024548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 936/1873] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:25.033425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815135-d415-4458-9294-8709305be5e8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.033719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815135-d415-4458-9294-8709305be5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.034000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815135-d415-4458-9294-8709305be5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.054318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d7815135-d415-4458-9294-8709305be5e8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:25.055898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d7815135-d415-4458-9294-8709305be5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:25.064032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 938/1874] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:25.068256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.077986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef94e-243f-400a-a58d-0cc9ec9acdd0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.082863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.083514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.083749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.084080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.084670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:25.084978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.086607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.089638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:25.102751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 937/1875] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:25.106067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815135-d415-4458-9294-8709305be5e8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.116064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815135-d415-4458-9294-8709305be5e8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.121518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.122157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.122462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.122742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.123426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.125624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.159278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 939/1876] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:25.160839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.173711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6150-3db5-4737-b387-5179f9de33a9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.244848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.245745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.246089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.246439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.247165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.249196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.284478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 938/1877] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:25.285878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.316113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63aa95-97d5-4da0-a816-7a57307e8088 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.328526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.329158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.329443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.329721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.330379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.332418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.345134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 940/1878] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:25.349316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.362017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d74b9-9cb7-4751-a49a-f768506ab23c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.368035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.369127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.369460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.369796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.372304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.374371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823a710-134e-40de-bcd3-bb5d8fc16bbb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.394172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 939/1879] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:25.395608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.413107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec4c2d-18ee-4182-bcee-65fce679576d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.420577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.422129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.422838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.423213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] PATH_INFO: `/v3/projects/f512b3df3a72452eba40ebcb199402de/users/1c5684267df14a4a8c741e25171035aa/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.449069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f512b3df3a72452eba40ebcb199402de, user_id=1c5684267df14a4a8c741e25171035aa, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.453667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb98429-9919-42ed-8e20-8d51f3dbdd42 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.485759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 941/1880] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:25 2022] PUT /identity/v3/projects/f512b3df3a72452eba40ebcb199402de/users/1c5684267df14a4a8c741e25171035aa/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:25.488906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.489329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.490301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.520473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf439-d785-4ff5-b728-6c77a8e0813f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.520611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf439-d785-4ff5-b728-6c77a8e0813f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.520899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf439-d785-4ff5-b728-6c77a8e0813f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.522197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 942/1881] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:43:25.532142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] MFA Rules not processed for user `1c5684267df14a4a8c741e25171035aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:25.533384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:25.535409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.535775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.536109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.564564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:25.565860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:25.649934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:25.665761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 940/1882] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:25.665761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.690190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 943/1883] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:25.704833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.706593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:25.721710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0a86c-4b6a-4761-971d-bfc19d53806b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.728748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.729372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.729654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.729926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.739992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.741602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.744424 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b927c5-df0b-4523-b45e-2852ca9f51ec None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.750321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.751143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.751535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.751875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.752587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.754789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.764443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 944/1884] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:25.776809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:25.804546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 941/1885] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:25.805896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.820625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cdfa-01c4-4db8-a4dd-9d6ced4f0d7d service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.888176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.889331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.889613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.889885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.890554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.893210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.914789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 945/1886] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:25 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:25.918631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.929126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74636ba-4ff4-4637-860f-638bac21e854 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.934669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.935276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.936190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.936456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.937099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.938760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:25.949109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 942/1887] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:25.954091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:25.964764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47a9c1-20cd-4a0a-84d1-ef1799903675 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:25.969561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:25.970361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:25.970675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:25.970952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] PATH_INFO: `/v3/projects/e7e6cd100b674c8a92560fa96af2105a/users/253b76d71dc84cc7b991002e8a17ee11/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:25.986570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e6cd100b674c8a92560fa96af2105a, user_id=253b76d71dc84cc7b991002e8a17ee11, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:25.989315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.006224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 946/1888] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:25 2022] PUT /identity/v3/projects/e7e6cd100b674c8a92560fa96af2105a/users/253b76d71dc84cc7b991002e8a17ee11/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:26.009931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.020221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7342064-96df-4fec-8cb8-f84cc62e5299 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.076510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.077149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.077449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.077721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.078358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.079982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317192ea-3cf2-4331-880f-5a434b7f77fe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.090152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 943/1889] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:26.094752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.104984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e6b3-b147-4726-909f-5084b060ee11 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.110192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.110963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.111245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.111526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] PATH_INFO: `/v3/projects/e7e6cd100b674c8a92560fa96af2105a/users/253b76d71dc84cc7b991002e8a17ee11/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.113439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e6cd100b674c8a92560fa96af2105a, user_id=253b76d71dc84cc7b991002e8a17ee11, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.116154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.127167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 947/1890] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:26 2022] PUT /identity/v3/projects/e7e6cd100b674c8a92560fa96af2105a/users/253b76d71dc84cc7b991002e8a17ee11/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:26.137956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.138240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.138654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.168348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] MFA Rules not processed for user `253b76d71dc84cc7b991002e8a17ee11`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:26.169619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:26.237455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 944/1891] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:26.250747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.252425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:26.309583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ac016-a4a3-43e5-99eb-be647de7eb1f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.313773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.314373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.314682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.314955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.323840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.325260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.334634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 948/1892] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:26.682990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.692696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853af9c4-c8dd-4b64-8854-730ab7c3ab04 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.697260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '253b76d71dc84cc7b991002e8a17ee11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7e6cd100b674c8a92560fa96af2105a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.697858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.698139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.698466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.699090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:26.699396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.701511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.712789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 945/1893] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:26.717577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.726676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b208-f610-4c0e-8281-65ec7d3ec71f service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.731265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '253b76d71dc84cc7b991002e8a17ee11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7e6cd100b674c8a92560fa96af2105a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.731861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.732148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.732425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.733062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.735002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.761773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 949/1894] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:26 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:26.766658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.776536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad333514-bad2-4978-800e-b046eda94235 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.819792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:26.834270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '253b76d71dc84cc7b991002e8a17ee11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7e6cd100b674c8a92560fa96af2105a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.834922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.835210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.835490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.836139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:26.836448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.838084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.850872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 946/1895] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:26.907124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:26.913988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:26.926813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a6d2-6d51-4d09-8060-3aa46f2752e0 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:26.935045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '253b76d71dc84cc7b991002e8a17ee11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7e6cd100b674c8a92560fa96af2105a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:26.935905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:26.936285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:26.936653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] PATH_INFO: `/v3/projects/468cb879ac794d6c9524af6c8cd2d7e2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:26.937821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorizing `identity:delete_project(project_id=468cb879ac794d6c9524af6c8cd2d7e2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:26.939826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:26.988485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468cb879ac794d6c9524af6c8cd2d7e2', 'request_id': 'req-9b215dc8-2c00-478f-8adc-b3eca87a2571'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:26.996344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468cb879ac794d6c9524af6c8cd2d7e2', 'request_id': 'req-9b215dc8-2c00-478f-8adc-b3eca87a2571'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.004083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468cb879ac794d6c9524af6c8cd2d7e2', 'request_id': 'req-9b215dc8-2c00-478f-8adc-b3eca87a2571'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.015085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] The token cache is being invalidate because project 468cb879ac794d6c9524af6c8cd2d7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:27.015375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.016167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9b215dc8-2c00-478f-8adc-b3eca87a2571 tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.017851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 950/1896] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:26 2022] DELETE /identity/v3/projects/468cb879ac794d6c9524af6c8cd2d7e2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:27.221179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:27.222918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:27.288316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e413-6ba7-480b-ad04-5164b2cdcc2d tempest-VolumeQuotaClassesTest-1450377880 tempest-VolumeQuotaClassesTest-1450377880-project-admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:27.343336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:27.343955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:27.344338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:27.344507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] PATH_INFO: `/v3/users/1c5684267df14a4a8c741e25171035aa` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:27.345596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5684267df14a4a8c741e25171035aa)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:27.347519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:27.391276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5684267df14a4a8c741e25171035aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.397851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5684267df14a4a8c741e25171035aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.398529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5684267df14a4a8c741e25171035aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.399126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5684267df14a4a8c741e25171035aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.406679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5684267df14a4a8c741e25171035aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.423169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 947/1897] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:27 2022] DELETE /identity/v3/users/1c5684267df14a4a8c741e25171035aa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:27.427643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:27.427932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:27.428237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:27.448582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:27.449794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:27.509711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 951/1898] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:27.519912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:27.521434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:27.531327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:27.592907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bb84-8d23-4cc8-bdb5-00832f66e312 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:27.596063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:27.596668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:27.596954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:27.597227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:27.606664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:27.608624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:27.618192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 948/1899] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:27.897166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:27.898771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:27.912824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d74746-2781-4bfc-9060-2aaa68063734 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:27.917964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:27.918625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:27.919456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:27.920551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] PATH_INFO: `/v3/projects/f512b3df3a72452eba40ebcb199402de` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:27.921717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f512b3df3a72452eba40ebcb199402de)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:27.923742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:27.974495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f512b3df3a72452eba40ebcb199402de', 'request_id': 'req-562dd950-092d-431f-84cf-582eb7b7bc4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.981967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f512b3df3a72452eba40ebcb199402de', 'request_id': 'req-562dd950-092d-431f-84cf-582eb7b7bc4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.988567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f512b3df3a72452eba40ebcb199402de', 'request_id': 'req-562dd950-092d-431f-84cf-582eb7b7bc4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:27.999579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] The token cache is being invalidate because project f512b3df3a72452eba40ebcb199402de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:27.999869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.000501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.001979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 952/1900] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:27 2022] DELETE /identity/v3/projects/f512b3df3a72452eba40ebcb199402de => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:28.006100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.016204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac211a-5afa-4cf7-bfa7-0efabababae4 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.074118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.074726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.075003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.075270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] PATH_INFO: `/v3/users/253b76d71dc84cc7b991002e8a17ee11` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.076531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253b76d71dc84cc7b991002e8a17ee11)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.080891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.127736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253b76d71dc84cc7b991002e8a17ee11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.135153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253b76d71dc84cc7b991002e8a17ee11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.135869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253b76d71dc84cc7b991002e8a17ee11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.136469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253b76d71dc84cc7b991002e8a17ee11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.144367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7685e012-e1d4-40fd-a1eb-3f6f87387b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253b76d71dc84cc7b991002e8a17ee11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.162978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 949/1901] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:28 2022] DELETE /identity/v3/users/253b76d71dc84cc7b991002e8a17ee11 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:28.442030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.444106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:28.463525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd950-092d-431f-84cf-582eb7b7bc4f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.472862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:28.528286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.528921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.529766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.530043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] PATH_INFO: `/v3/projects/e7e6cd100b674c8a92560fa96af2105a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.531209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e6cd100b674c8a92560fa96af2105a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.533157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.580001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e6cd100b674c8a92560fa96af2105a', 'request_id': 'req-9774e434-3e18-407d-858f-088e32ae0e94'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.587232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e6cd100b674c8a92560fa96af2105a', 'request_id': 'req-9774e434-3e18-407d-858f-088e32ae0e94'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.593537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e6cd100b674c8a92560fa96af2105a', 'request_id': 'req-9774e434-3e18-407d-858f-088e32ae0e94'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.603642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] The token cache is being invalidate because project e7e6cd100b674c8a92560fa96af2105a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:28.603937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.604574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9774e434-3e18-407d-858f-088e32ae0e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:28.606088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 953/1902] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:28 2022] DELETE /identity/v3/projects/e7e6cd100b674c8a92560fa96af2105a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:28.610937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.611397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.611506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.631325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:28.632592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:28.665116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:28.703188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 950/1903] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:28.710856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.711275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.711651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.733736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:28.734963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:28.742682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 954/1904] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:28.746919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.756830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2bc7-d84f-4a2e-86ec-0d730c6ed36f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.761722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.762327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.762653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.762923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.763611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:28.763912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.765500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.778479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 951/1905] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:28.783294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.792563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358301-921c-4767-a9b0-034dbbdcd3d3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.798069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.798675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.798949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.799217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.799879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.801795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.842480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 955/1906] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:28 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:28.846785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.856774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5edaa-3f2e-4752-8fbf-79c64e723cd1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.914709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.915349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.915629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.915914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.916557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.918595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.940578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 952/1907] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:28 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:28.944745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.957647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9d98c-a02c-4775-949b-ae9c92975be1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.962061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.962685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.962963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.963233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.963867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.965479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.975454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 956/1908] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:28.979710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.992814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032799b-b26f-4f59-8328-d90446634a6e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:29.012521 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:29.013333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:29.013675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:29.014182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:29.014934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:29.016982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702f95e-d641-437b-9c85-7a3a8f53aac4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:29.029127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 953/1909] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:29.033182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:29.053325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ad31-ff62-4b21-a1ff-fb633f1ed621 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:29.058003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:29.058830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:29.059129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:29.059399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] PATH_INFO: `/v3/projects/b7fc79cd5d144cec991c96c4e72ec0a8/users/b410ecaf96f44dd2bc9c89f82a0099bc/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:29.076770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7fc79cd5d144cec991c96c4e72ec0a8, user_id=b410ecaf96f44dd2bc9c89f82a0099bc, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:29.080083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:29.096506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 957/1910] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:29 2022] PUT /identity/v3/projects/b7fc79cd5d144cec991c96c4e72ec0a8/users/b410ecaf96f44dd2bc9c89f82a0099bc/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:29.100147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0cf3-0ea1-4e85-98f5-17aa631c6de7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:29.100434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0cf3-0ea1-4e85-98f5-17aa631c6de7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:29.100754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0cf3-0ea1-4e85-98f5-17aa631c6de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:29.122090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-549d0cf3-0ea1-4e85-98f5-17aa631c6de7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:29.124046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-549d0cf3-0ea1-4e85-98f5-17aa631c6de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:29.202475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 954/1911] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:29.216548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:29.218344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:29.290284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:29.313115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bec9c-2b42-4efc-98c6-7be6fc6e604d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:29.316942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:29.317569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:29.317852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:29.318342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:29.328737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:29.330240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:29.343025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 958/1912] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:30.094101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.094638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.094727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.129349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] MFA Rules not processed for user `b410ecaf96f44dd2bc9c89f82a0099bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:30.130590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:30.154925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:30.199069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 955/1913] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:30.204296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.214176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f0074-561d-4633-838e-6438dd132354 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.218898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.219499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.219781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.220054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.220695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.222628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.257944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 959/1914] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:30 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:30.262016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.273687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db50db-10c2-40ae-94a6-40c3c34083d8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.331616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.332242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.332536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.332822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.333466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.336376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.358907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 956/1915] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:30 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:30.362943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.373532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc919-5890-45fc-a95d-feb5c951d00c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.378113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.378750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.379037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.379304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.379946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.381542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.392013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 960/1916] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:30.396128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.406603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c688-e60b-425e-b43f-a1dbea51f7d5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.411289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.411942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.412223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.412499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] PATH_INFO: `/v3/projects/ff09cf986a604e03bc565a7df6a4e002/users/e8fbf91c5a4d4ac9a742816831a353a7/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.427922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff09cf986a604e03bc565a7df6a4e002, user_id=e8fbf91c5a4d4ac9a742816831a353a7, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.430615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.444038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 957/1917] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:30 2022] PUT /identity/v3/projects/ff09cf986a604e03bc565a7df6a4e002/users/e8fbf91c5a4d4ac9a742816831a353a7/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:30.447971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.458372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ef05-6e16-4721-9ba5-f1809c0d5049 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.523244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.524377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.524662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.527201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.528125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.531254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dca129-9968-40d1-9180-7ed67668e09a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.546306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 961/1918] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:30.551242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:30.562402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421fc2-6796-4b46-989b-953a57de6822 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:30.567277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:30.568002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:30.568356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:30.568706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] PATH_INFO: `/v3/projects/ff09cf986a604e03bc565a7df6a4e002/users/e8fbf91c5a4d4ac9a742816831a353a7/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:30.570799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff09cf986a604e03bc565a7df6a4e002, user_id=e8fbf91c5a4d4ac9a742816831a353a7, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:30.573462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:30.584643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 958/1919] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:30 2022] PUT /identity/v3/projects/ff09cf986a604e03bc565a7df6a4e002/users/e8fbf91c5a4d4ac9a742816831a353a7/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:31.556622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:31.557183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:31.557183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:31.588930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] MFA Rules not processed for user `e8fbf91c5a4d4ac9a742816831a353a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:31.590172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:31.673167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 962/1920] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:31.684670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:31.686487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:31.753332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c2bb9-cf89-43e7-949c-284e2f530130 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:31.757061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:31.757661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:31.757940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:31.758210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:31.790529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:31.828277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:31.829755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cc9f8-a90c-450b-9f27-598f3d5d82c4 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:31.839254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 959/1921] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:32.170675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:32.172071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:32.240602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:32.287189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea7b15-de42-4551-9aa4-93adef52fdd4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:32.291828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:32.292465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:32.292743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:32.293215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] PATH_INFO: `/v3/users/b638d111d9c34b4c80f4370aea97eeff` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:32.294149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b638d111d9c34b4c80f4370aea97eeff)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:32.296534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:32.365619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b638d111d9c34b4c80f4370aea97eeff'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:32.373593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b638d111d9c34b4c80f4370aea97eeff'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:32.374299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b638d111d9c34b4c80f4370aea97eeff'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:32.374965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b638d111d9c34b4c80f4370aea97eeff'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:32.385057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b638d111d9c34b4c80f4370aea97eeff'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:32.406455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 963/1922] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:32 2022] DELETE /identity/v3/users/b638d111d9c34b4c80f4370aea97eeff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:32.408961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:32.409251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:32.410441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:32.428948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:32.430284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:32.498251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 960/1923] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:32.508279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:32.509882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:32.591180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d2b53-11c5-4bac-ae20-31c03c24aa89 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:32.594854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:32.595463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:32.595745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:32.596020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:32.605384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:32.606872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da11c76-3741-491c-ba1f-a1236ca99279 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:32.616415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 964/1924] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:33.042679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.044227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:33.057781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e755b66-9f57-487e-9da6-b1c9028e01c8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.064242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.065082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.065366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.065686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] PATH_INFO: `/v3/projects/fb33cd0774ee4ae8992645c1be7554ff` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.066849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb33cd0774ee4ae8992645c1be7554ff)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.068698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.115242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb33cd0774ee4ae8992645c1be7554ff', 'request_id': 'req-c0b09520-6690-44a0-a0f3-f9ce942f7c87'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:33.120832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb33cd0774ee4ae8992645c1be7554ff', 'request_id': 'req-c0b09520-6690-44a0-a0f3-f9ce942f7c87'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:33.129150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb33cd0774ee4ae8992645c1be7554ff', 'request_id': 'req-c0b09520-6690-44a0-a0f3-f9ce942f7c87'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:33.141411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] The token cache is being invalidate because project fb33cd0774ee4ae8992645c1be7554ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:33.141700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:33.142384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c0b09520-6690-44a0-a0f3-f9ce942f7c87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:33.143898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 961/1925] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:33 2022] DELETE /identity/v3/projects/fb33cd0774ee4ae8992645c1be7554ff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:33.148714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.148994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.149262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.168228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:33.169338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:33.213964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:33.239506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 965/1926] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:33.245740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.245740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.245740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.281474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:33.282693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:33.291077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 962/1927] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:33.295212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.305096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904e61-c9dc-4ec5-9b7d-19a37733977d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.311390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.311999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.312277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.312550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.313206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:33.313501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.315132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.331600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 966/1928] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:33.335920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.345315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c496e-ed74-4ddc-9a3f-b3ec11e335b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.349955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.350598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.350823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.351091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.351718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.354338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.381374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 963/1929] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:33.385519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.396030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a62f2-6201-460a-b5de-1c291cec8d5a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.451973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.452504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.452823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.453096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.453734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.455743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.477219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 967/1930] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:33.481737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.492661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecb08-1457-4fc6-94b7-d9af5b4849bd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.497335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.498582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.498863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.499139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.499789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.501388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.511142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 964/1931] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:33.515089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.524891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdcf6d-0627-4dd2-9d56-1f79c875c877 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.529661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.530253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.530576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.530850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.531497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.533090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099bbaa0-e2f3-4598-ac9b-144917bef3e5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.543876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 968/1932] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:33.549739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.560772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255d63-bdc4-4e6c-8cae-85955c2cdd53 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.565433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.566228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.568119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.568412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] PATH_INFO: `/v3/projects/0f07861ca014450280f968c4cd49cacc/users/096dd6902db149369f5963f32d5eeb48/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.581691 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:33.585824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f07861ca014450280f968c4cd49cacc, user_id=096dd6902db149369f5963f32d5eeb48, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.592003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.611271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 965/1933] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:33 2022] PUT /identity/v3/projects/0f07861ca014450280f968c4cd49cacc/users/096dd6902db149369f5963f32d5eeb48/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:33.621575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.622137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.622137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.653844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] MFA Rules not processed for user `096dd6902db149369f5963f32d5eeb48`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:33.655188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:33.738954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 969/1934] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:33.743434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.818536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70a9ab-6e7d-47c7-807f-6c34921d755f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.823141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.823742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.824015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.824284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.824906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.827393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.853305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 966/1935] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:33.858143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.868729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6380e1c-2c58-4720-95d8-5eaae1817a5c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.911106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:33.927314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.927920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.928203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.928527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.929128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.931075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.955011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 970/1936] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:33 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:33.959460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:33.970807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d289-cc18-4e5f-a287-4da65438c6e8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:33.975405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:33.976001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:33.976294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:33.976553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:33.977196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:33.978835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:33.990148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 967/1937] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:33.994101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.004344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5381ba-1b31-4e0d-b215-0e6c44ffa388 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.009015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.009609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.009892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.010164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.010848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.012441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0b40-8495-4827-a5a0-9dcee7ce9dc5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.022232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 971/1938] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:34.026323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.036830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71dc-4311-47a8-bf55-45624a133458 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.041407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.042047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.042328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.042631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] PATH_INFO: `/v3/projects/a6e50051df0c4900842b834c0ef0c00b/users/69ee174a9dd94a9387d56d0f0aeb9bb3/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.057468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e50051df0c4900842b834c0ef0c00b, user_id=69ee174a9dd94a9387d56d0f0aeb9bb3, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.060433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.073461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 968/1939] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:34 2022] PUT /identity/v3/projects/a6e50051df0c4900842b834c0ef0c00b/users/69ee174a9dd94a9387d56d0f0aeb9bb3/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:34.084088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.084378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.084655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.113432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] MFA Rules not processed for user `69ee174a9dd94a9387d56d0f0aeb9bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:34.114791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:34.178085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 972/1940] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:34.182829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.244689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4083d12-7a9f-45b9-893d-5dbaa921d6c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.249273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.249909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.250192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.250492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.251138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.253077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.283403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 969/1941] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:34 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:34.288040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.299185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10db8ae-9ce8-494a-a09c-4c65277e8a2c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.358828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.359616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.359713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.359965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.360600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.362527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.386278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 973/1942] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:34 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:34.391007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.406889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac11e-3d39-4606-987f-101cd4dbb9a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.411677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.412360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.412646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.412923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.413603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.415231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.426102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 970/1943] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:34.430274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.440443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101582-15d7-4a9f-941f-cd4c8a53c885 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.444996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.445619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.445901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.446174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] PATH_INFO: `/v3/projects/3c2cd65e62324421abd1cac27aa850e7/users/89eee876aacd400f8d9ac389ce170396/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.461152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2cd65e62324421abd1cac27aa850e7, user_id=89eee876aacd400f8d9ac389ce170396, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.463849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.478971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 974/1944] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:34 2022] PUT /identity/v3/projects/3c2cd65e62324421abd1cac27aa850e7/users/89eee876aacd400f8d9ac389ce170396/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:34.483152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.492928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee13b6-441b-4bf8-8df7-29b4b7bf3452 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.504367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:34.550061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.550686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.550966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.551263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.551888 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.553463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc781fba-c247-43b1-b29a-8873149b9c97 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.563751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 971/1945] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:34.567886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.578617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d000-59b3-47da-b8fb-f99709ce1f4d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.583128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.583767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.584046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.584318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] PATH_INFO: `/v3/projects/3c2cd65e62324421abd1cac27aa850e7/users/89eee876aacd400f8d9ac389ce170396/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.586225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2cd65e62324421abd1cac27aa850e7, user_id=89eee876aacd400f8d9ac389ce170396, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.588901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.598902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 975/1946] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:34 2022] PUT /identity/v3/projects/3c2cd65e62324421abd1cac27aa850e7/users/89eee876aacd400f8d9ac389ce170396/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:34.610094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.610385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.610688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.638902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] MFA Rules not processed for user `89eee876aacd400f8d9ac389ce170396`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:34.639818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:34.706748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 972/1947] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:34.717870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.719604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:34.726137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:34.788604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe247e1-a5c8-4f29-a7c4-f38f116c1b29 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.792469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.793071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.793349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.793630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.866327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.867823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.891389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 976/1948] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:38.617150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:38.618607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:38.631168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bebd-dcd3-4817-93c5-bc22f8ccd730 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:38.635908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:38.636542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:38.636864 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:38.637210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:38.639827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:38.641568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:38.652086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b2a-21a6-44f1-b3d5-b58c9c6b0a02 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:38.656383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:38.656996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:38.657271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:38.657544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:38.723407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:38.725338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c398a9e-754a-4769-b61e-e77b8e6a0350 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:38.729692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:38.731267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57106d4-e809-4c9b-b192-575fb5addee0 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:38.741128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 973/1949] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:38.742324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 977/1950] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:39.105430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.106703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.107988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.110151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 974/1951] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:43:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:43:39.133940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c029ed-078c-4605-8818-49541c94125f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.134311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c029ed-078c-4605-8818-49541c94125f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.134513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c029ed-078c-4605-8818-49541c94125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.182111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-21c029ed-078c-4605-8818-49541c94125f None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:39.184763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-21c029ed-078c-4605-8818-49541c94125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:39.271150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:39.294894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 978/1952] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:39.308541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:39.310751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:39.331244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ff661-7c83-4c5e-8279-17ec1d89f0da None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:39.337891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:39.338556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.338837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.339120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.349057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:39.350690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:39.363536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 975/1953] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:39.666216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c029ed-078c-4605-8818-49541c94125f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:39.667757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-21c029ed-078c-4605-8818-49541c94125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:39.759066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:39.764319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c029ed-078c-4605-8818-49541c94125f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:39.770667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:39.771800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.772089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.772205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531eb498-74d2-44dd-a971-bccc30f1f319 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:39.772368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] PATH_INFO: `/v3/users/bd4f1605139f4a239f51dcb3c25156c8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.773495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4f1605139f4a239f51dcb3c25156c8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:39.775737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:39.775921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:39.776331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.776609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.776882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.788007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:39.789631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5f364-17a0-45fa-9cb1-4119d917adf6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:39.801718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 976/1954] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:39.840135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4f1605139f4a239f51dcb3c25156c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:39.849338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4f1605139f4a239f51dcb3c25156c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:39.850105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4f1605139f4a239f51dcb3c25156c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:39.850748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4f1605139f4a239f51dcb3c25156c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:39.863524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4f1605139f4a239f51dcb3c25156c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:39.894166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 979/1955] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:39 2022] DELETE /identity/v3/users/bd4f1605139f4a239f51dcb3c25156c8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:39.902564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:39.902850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:39.903130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:39.911923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:39.927662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:39.928949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:40.036172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 977/1956] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:40.047602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:40.050023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:40.127071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996376f2-6abc-4994-ae91-f3e68bc80ece admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:40.130729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:40.131345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:40.131627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:40.131902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:40.142111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:40.143591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:40.153418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 980/1957] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:40.591914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:40.593464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:40.606016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f83d-a0cf-4898-8058-d2ee8beeeba8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:40.611434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:40.612174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:40.612526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:40.612876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] PATH_INFO: `/v3/projects/625adb5fa2a54a029c1c5326340a1ccf` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:40.614066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] RBAC: Authorizing `identity:delete_project(project_id=625adb5fa2a54a029c1c5326340a1ccf)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:40.616080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:40.675274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625adb5fa2a54a029c1c5326340a1ccf', 'request_id': 'req-6c6dffd4-1d83-4de6-85c6-bda5913e505f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.682836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625adb5fa2a54a029c1c5326340a1ccf', 'request_id': 'req-6c6dffd4-1d83-4de6-85c6-bda5913e505f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.691744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625adb5fa2a54a029c1c5326340a1ccf', 'request_id': 'req-6c6dffd4-1d83-4de6-85c6-bda5913e505f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.707061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] The token cache is being invalidate because project 625adb5fa2a54a029c1c5326340a1ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:40.707317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.708015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.709728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 978/1958] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:40 2022] DELETE /identity/v3/projects/625adb5fa2a54a029c1c5326340a1ccf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:40.713877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:40.725497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047235-e160-41d6-9810-80120758e6fa service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:40.735494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:40.838777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:40.839402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:40.839689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:40.839962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] PATH_INFO: `/v3/users/d1d492e9d4974af682f7ffef161d80cb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:40.841079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d492e9d4974af682f7ffef161d80cb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:40.843040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:40.903861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d492e9d4974af682f7ffef161d80cb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.911064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d492e9d4974af682f7ffef161d80cb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.911741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d492e9d4974af682f7ffef161d80cb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.912340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d492e9d4974af682f7ffef161d80cb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.920317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c318bb3f-8623-47f6-8b21-37839b8c1448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d492e9d4974af682f7ffef161d80cb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:40.937994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 981/1959] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:40 2022] DELETE /identity/v3/users/d1d492e9d4974af682f7ffef161d80cb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:41.203501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.205004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:41.218630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dffd4-1d83-4de6-85c6-bda5913e505f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.279735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.281042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.281410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.281691 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] PATH_INFO: `/v3/projects/f7da70bbd8504cd697ef3cf97f968f68` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.282875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7da70bbd8504cd697ef3cf97f968f68)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.285309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.320115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:41.334471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7da70bbd8504cd697ef3cf97f968f68', 'request_id': 'req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:41.339854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7da70bbd8504cd697ef3cf97f968f68', 'request_id': 'req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:41.347908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7da70bbd8504cd697ef3cf97f968f68', 'request_id': 'req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:41.359600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] The token cache is being invalidate because project f7da70bbd8504cd697ef3cf97f968f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:41.359887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:41.360557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c848e7f5-ff5e-4b3b-8af8-857bb48fe1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:41.362027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 979/1960] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:41 2022] DELETE /identity/v3/projects/f7da70bbd8504cd697ef3cf97f968f68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:41.366930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.367218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.367511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.386824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:41.388082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:41.446547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 982/1961] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:41.454459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.454751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.455028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.474824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:41.476021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:41.484382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 980/1962] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:41.488562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.498081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cf81a-c221-48e9-a983-cfe008545c4e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.502597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.503184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.503460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.503728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.504364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:41.504664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.506258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.517559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 983/1963] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:41.521964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.531910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954fba7-b9ac-4c9d-9cc6-dc731bc7a429 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.545596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.546558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.546965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.547304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.548347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.550371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.580844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 981/1964] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:41.585062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.596346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83482-e70b-4f35-aade-9aa4ceeb253d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.610675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:41.661482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.662750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.663024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.663294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.663939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.665877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.693723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 984/1965] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:41.694961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.708487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46463-7244-42a0-9b25-35cbd3b54763 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.713866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.714507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.714820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.715239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.716719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.718312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.728442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 982/1966] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:41.732659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.743045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329778b6-3f08-4eb5-b537-7578adb0450a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.747759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.748579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.749020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.749444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.750260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.751893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4e04b-c11c-453c-8f6b-8ec41b909e4f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.770025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 985/1967] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:41.771942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:41.782800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dead6c6-dba3-4716-8459-0eefa20d3dd6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:41.787508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:41.788103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.788384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.788658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] PATH_INFO: `/v3/projects/fbf5aac8d5e64a41b920aef35307af30/users/a065e6279bca4930becd4eb6c7254721/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.803833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf5aac8d5e64a41b920aef35307af30, user_id=a065e6279bca4930becd4eb6c7254721, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:41.806512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:41.821499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 983/1968] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:41 2022] PUT /identity/v3/projects/fbf5aac8d5e64a41b920aef35307af30/users/a065e6279bca4930becd4eb6c7254721/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:41.833340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:41.833622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:41.833897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:41.861783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] MFA Rules not processed for user `a065e6279bca4930becd4eb6c7254721`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:41.862992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:41.931794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 986/1969] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:41.936873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.008745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bef6e-6bfd-45e5-92bb-1297dcf6cb31 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.014392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.015205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.015489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.015788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.016425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.018354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.052816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 984/1970] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:41 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:42.054683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.079864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec1938-02cb-44f4-9fce-f51c9c4ac7b3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.135079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.135681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.135956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.136228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.136955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.139464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.159874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 987/1971] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:42 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:42.163875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.174229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5202791-7254-49d0-922c-cb648dda57fa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.178899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.179503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.179781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.180053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.180690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.182273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.193740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 985/1972] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:42.197953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.208326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949790f-1aa8-43de-8c16-61203870323a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.212860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.213495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.213771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.214040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] PATH_INFO: `/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac/users/fb59a329df724ac68958d686bb453a66/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.225660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:42.229060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba6a0e9b8f1488b93df8f907eda31ac, user_id=fb59a329df724ac68958d686bb453a66, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.232205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.250339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 988/1973] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:42 2022] PUT /identity/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac/users/fb59a329df724ac68958d686bb453a66/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:42.250538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.260228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81352fa-5e32-4e86-bef4-94260e509ea3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.286881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:42.321079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.321597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.321930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.322245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.322927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.324530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53f532-66ee-40e3-ac5f-62d409ead038 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.334995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 986/1974] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:42.339173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.349337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539295-c597-44c4-a2ae-2d6b9a5778f4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.353845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.354511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.354794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.355066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] PATH_INFO: `/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac/users/fb59a329df724ac68958d686bb453a66/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.356940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba6a0e9b8f1488b93df8f907eda31ac, user_id=fb59a329df724ac68958d686bb453a66, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.359597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.369550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 989/1975] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:42 2022] PUT /identity/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac/users/fb59a329df724ac68958d686bb453a66/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:42.380543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.380837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.381115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.409295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] MFA Rules not processed for user `fb59a329df724ac68958d686bb453a66`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:42.410528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:42.477086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 987/1976] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:42.489023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.490769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:42.546926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe6a7e-c0c1-42db-82bd-824b613ca56a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.550552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.551148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.551429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.551703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.561427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.562891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.572303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 990/1977] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:42.661112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:42.670977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368dca7-da4e-4262-9aa2-15b9ddff8ed6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:42.674522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:42.675113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:42.675390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:42.675678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:42.731967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:42.733397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a75f3-25d3-44f6-a97f-6812ca6e5080 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:42.742152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 988/1978] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:43.060213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:43.135155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa6a6-b38b-4eba-a020-6438fe373be3 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:43.140321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:43.140949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:43.141230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:43.141504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] PATH_INFO: `/v3/users/a065e6279bca4930becd4eb6c7254721` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:43.142655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a065e6279bca4930becd4eb6c7254721)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:43.144702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:43.190805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a065e6279bca4930becd4eb6c7254721'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:43.197657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a065e6279bca4930becd4eb6c7254721'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:43.198337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a065e6279bca4930becd4eb6c7254721'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:43.198964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a065e6279bca4930becd4eb6c7254721'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:43.207466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a065e6279bca4930becd4eb6c7254721'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:43.220527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:43.225470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 991/1979] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:43 2022] DELETE /identity/v3/users/a065e6279bca4930becd4eb6c7254721 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:43.230384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:43.230701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:43.230978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:43.251050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:43.252248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:43.297049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:43.313506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 989/1980] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:43.323393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:43.324851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:43.394735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152da86-33fa-4f5c-8da4-fd541a54e8d4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:43.399618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:43.400237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:43.400514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:43.400787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:43.410441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:43.411886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:43.422637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 992/1981] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:43.993956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:43.995363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:44.009501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e70c2-8742-49e3-a55b-94560a77becf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:44.015312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:44.016393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.016574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.016870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] PATH_INFO: `/v3/projects/fbf5aac8d5e64a41b920aef35307af30` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.018015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf5aac8d5e64a41b920aef35307af30)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:44.019954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:44.071315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf5aac8d5e64a41b920aef35307af30', 'request_id': 'req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.078250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf5aac8d5e64a41b920aef35307af30', 'request_id': 'req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.087834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf5aac8d5e64a41b920aef35307af30', 'request_id': 'req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.104106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] The token cache is being invalidate because project fbf5aac8d5e64a41b920aef35307af30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:44.104398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.105085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.106616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 990/1982] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:43 2022] DELETE /identity/v3/projects/fbf5aac8d5e64a41b920aef35307af30 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:44.110578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:44.120873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcba73-a3d2-4ea5-b280-afac302e52be service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:44.210983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:44.211626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.211908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.212185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] PATH_INFO: `/v3/users/fb59a329df724ac68958d686bb453a66` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.213422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb59a329df724ac68958d686bb453a66)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:44.215509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:44.267281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb59a329df724ac68958d686bb453a66'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.274484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb59a329df724ac68958d686bb453a66'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.275206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb59a329df724ac68958d686bb453a66'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.275845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb59a329df724ac68958d686bb453a66'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.285171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b097ac93-4651-44c6-9293-87e486a80d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb59a329df724ac68958d686bb453a66'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.308573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 993/1983] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:44 2022] DELETE /identity/v3/users/fb59a329df724ac68958d686bb453a66 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:44.613489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:44.615045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:44.630926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e65bf4-2df8-45be-b5aa-fbcbed1621d1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:44.695910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:44.696471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.696748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.697018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] PATH_INFO: `/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.698233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba6a0e9b8f1488b93df8f907eda31ac)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:44.700902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:44.749114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba6a0e9b8f1488b93df8f907eda31ac', 'request_id': 'req-cf2cf4c8-462e-4c20-b834-7560304c181f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.754747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba6a0e9b8f1488b93df8f907eda31ac', 'request_id': 'req-cf2cf4c8-462e-4c20-b834-7560304c181f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.764267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba6a0e9b8f1488b93df8f907eda31ac', 'request_id': 'req-cf2cf4c8-462e-4c20-b834-7560304c181f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.775708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] The token cache is being invalidate because project aba6a0e9b8f1488b93df8f907eda31ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:44.775989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.776630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf2cf4c8-462e-4c20-b834-7560304c181f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:44.778099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 991/1984] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:44 2022] DELETE /identity/v3/projects/aba6a0e9b8f1488b93df8f907eda31ac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:44.782988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.783286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.783568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.803214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:44.804432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:44.823822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:44.870521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 994/1985] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:44.878302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.878625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.878908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.886715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:44.903969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:44.905397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:44.924445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 992/1986] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:44.928453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:44.939665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8c412-51d1-4727-8f51-0792f9622f3c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:44.944733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:44.945531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.945816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.946149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.946787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:44.947111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:44.948702 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:44.963202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 995/1987] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:44.968618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:44.978173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c555eb1-73c5-4599-90c4-0f71e4821eda None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:44.983094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:44.983708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.983988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.984260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.984913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:44.987096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.023905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 993/1988] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:44 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:45.027816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.044256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac723-3de0-4ce5-a735-36a82d37fe0c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.124173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.125077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.125077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.125343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.125931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.127887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.150510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 996/1989] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:45 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:45.155117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.165467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409069b-bd2d-41f8-8642-f459b3c266cb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.170161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.170789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.171070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.171345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.171989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.173609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.183761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 994/1990] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:45.188293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.198330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b06fbd-de58-4ef1-8fb7-eb69c1119ac9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.203679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.204373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.204679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.204954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.205611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.207384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888e36c-f554-4cee-b361-4afc8d69ac0e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.221826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 997/1991] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:45.226172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.237007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb1c1-9c22-4861-a4cb-1d102ce884c6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.242070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.242816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.243241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.243525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] PATH_INFO: `/v3/projects/d228aab06f3c4f798519d2e782320844/users/9b1fa42080164e1ea55bd0dac3f63adb/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.282707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d228aab06f3c4f798519d2e782320844, user_id=9b1fa42080164e1ea55bd0dac3f63adb, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.286471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.303343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 995/1992] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:45 2022] PUT /identity/v3/projects/d228aab06f3c4f798519d2e782320844/users/9b1fa42080164e1ea55bd0dac3f63adb/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:45.317673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.317958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.318676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.364085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] MFA Rules not processed for user `9b1fa42080164e1ea55bd0dac3f63adb`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:45.365597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:45.500563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 998/1993] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:45.505381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.613617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d50350-bffe-4d62-95d6-f4d53cbb9e72 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.622711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.624095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.624510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.625534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.626548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.629131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.666770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 996/1994] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:45 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:45.678202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.691155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b2b65-ba16-4317-acad-0197abd44850 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.705994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:45.749940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.750554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.750835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.751105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.751738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.753676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.774966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 999/1995] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:45 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:45.779920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.789837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c062e-4d8b-4d91-a2e9-f938cb7cd840 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.794491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.795084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.795369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.795638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.796370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.797896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.808472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 997/1996] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:45.813284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.823802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7484f7-0011-4ec5-b866-1bd4e3a5d8b9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.829354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.830006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.830290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.830604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] PATH_INFO: `/v3/projects/e183ffbca4a04d2ab48f53332d3cd725/users/249adcc220fd4debb3dd537ccbebb957/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.845194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e183ffbca4a04d2ab48f53332d3cd725, user_id=249adcc220fd4debb3dd537ccbebb957, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.847903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.860610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1000/1997] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:45 2022] PUT /identity/v3/projects/e183ffbca4a04d2ab48f53332d3cd725/users/249adcc220fd4debb3dd537ccbebb957/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:45.864617 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.875060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffd27f-4fb3-4437-bec8-0715a0212cb0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.935301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.935914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.936196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.936467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.937101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.938715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df47de-0144-4031-ba6c-cb2334afcff0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.948403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 998/1998] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:45.953551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:45.964583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa10eb-d5ba-4281-a68f-26c14a16aa4e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:45.969210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:45.969856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.970135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.970432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] PATH_INFO: `/v3/projects/e183ffbca4a04d2ab48f53332d3cd725/users/249adcc220fd4debb3dd537ccbebb957/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:45.972347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e183ffbca4a04d2ab48f53332d3cd725, user_id=249adcc220fd4debb3dd537ccbebb957, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:45.975170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:45.985397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1001/1999] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:45 2022] PUT /identity/v3/projects/e183ffbca4a04d2ab48f53332d3cd725/users/249adcc220fd4debb3dd537ccbebb957/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:45.995688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:45.995979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:45.996260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:46.017408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:46.031370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] MFA Rules not processed for user `249adcc220fd4debb3dd537ccbebb957`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:46.032644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:46.055509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:46.057347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:46.108965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 999/2000] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:46.124174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:46.125782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:46.138263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c53ea-b7a8-4a5b-9013-2a46e8316a03 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:46.143160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:46.143805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:46.144245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:46.144524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] PATH_INFO: `/v3/users/532e66482b9d4ed9a33dff51fcf37421` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:46.145677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=532e66482b9d4ed9a33dff51fcf37421)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:46.147799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:46.206724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b118-182d-4a96-97d8-9441d2e71167 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:46.210581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:46.211320 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:46.211658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:46.211968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:46.213640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532e66482b9d4ed9a33dff51fcf37421'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:46.231232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532e66482b9d4ed9a33dff51fcf37421'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:46.231232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532e66482b9d4ed9a33dff51fcf37421'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:46.231232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532e66482b9d4ed9a33dff51fcf37421'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:46.239075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-618eee3f-8a70-471e-b719-75b033c22cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532e66482b9d4ed9a33dff51fcf37421'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:46.248820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:46.280300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1002/2001] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:46 2022] DELETE /identity/v3/users/532e66482b9d4ed9a33dff51fcf37421 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:46.284730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:46.285021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:46.285293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:46.287281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:46.288894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:46.301073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1000/2002] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:46.310938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:46.312018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:46.383648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1003/2003] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:46.395816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:46.397277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:46.459019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:46.463450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a650c-4597-42d1-b916-a28c131ca553 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:46.531407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:46.531750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:46.532103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:46.532449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:46.542673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:46.544688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e52fbd-4da7-4530-8e16-e5180d4b2afa service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:46.554592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1001/2004] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:47.117775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.119340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:47.141732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547d02-c1a7-4dd7-b182-4f0f5d775c8f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.180237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.181503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.181782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.182056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] PATH_INFO: `/v3/projects/64077172128249a1bf0d8b58da80a450` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.183625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64077172128249a1bf0d8b58da80a450)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.191670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.320137 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64077172128249a1bf0d8b58da80a450', 'request_id': 'req-0c667f38-66f5-47fd-a210-ec095783fe9f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:47.336175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64077172128249a1bf0d8b58da80a450', 'request_id': 'req-0c667f38-66f5-47fd-a210-ec095783fe9f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:47.346107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64077172128249a1bf0d8b58da80a450', 'request_id': 'req-0c667f38-66f5-47fd-a210-ec095783fe9f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:47.364317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] The token cache is being invalidate because project 64077172128249a1bf0d8b58da80a450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:47.365006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:47.366751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0c667f38-66f5-47fd-a210-ec095783fe9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:47.368737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1004/2005] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:47 2022] DELETE /identity/v3/projects/64077172128249a1bf0d8b58da80a450 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:47.374544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.375022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.375107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.407840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:47.409199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:47.492839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1002/2006] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:47.500312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.500814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.501325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.523437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:47.524750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:47.539262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1005/2007] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:47.540831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.555834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102618-a7bf-4753-901a-b633542e79eb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.563375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.564012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.564270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.564540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.565197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:47.565500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.567151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.579316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1003/2008] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:47.583635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.593493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0830daa-0e7d-4a11-b668-5754b2649149 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.598199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.598828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.599108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.599380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.600027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.601923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.630880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1006/2009] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:47 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:47.635327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.645182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664b5a3-978b-4bd8-ae34-0225d6f6a519 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.722825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.723477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.724202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.724202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.726229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.729850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.777224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1004/2010] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:47 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:47.782532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.795857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6300-4808-41c0-9c02-1d0090387019 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.802074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.803020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.803441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.803824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.804809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.807141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.821896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1007/2011] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:47.824960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.835408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc832-0901-454d-8431-2154880aa408 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.840792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.841908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.842184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.842479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.843123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.844729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e76d-936e-44cc-9c0a-f0050b4f77ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.856518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1005/2012] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:47.860683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:47.872264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc04c4-f72f-411b-aaaa-8c106116d79d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:47.877247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:47.878033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.878384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.878761 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] PATH_INFO: `/v3/projects/c366db70acfd403a80504ad1abffaeb4/users/0e2dcf7120cb4786b59798979738424e/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.895813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c366db70acfd403a80504ad1abffaeb4, user_id=0e2dcf7120cb4786b59798979738424e, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:47.898665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:47.913764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1008/2013] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:47 2022] PUT /identity/v3/projects/c366db70acfd403a80504ad1abffaeb4/users/0e2dcf7120cb4786b59798979738424e/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:47.925748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:47.926109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:47.926480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:47.956940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:47.960839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] MFA Rules not processed for user `0e2dcf7120cb4786b59798979738424e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:47.962902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:48.026152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1006/2014] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:48.031340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.041473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:48.096657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0bf7-ceed-4d43-9551-c867e113a94c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.101215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.101814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.102097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.102369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.103044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.104945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.136633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1009/2015] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:48 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:48.142791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.155363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286a45d-b5f5-406e-9ff1-06526c7fd513 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.238960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.239800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.240176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.240516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.241230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.243307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.270186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1007/2016] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:48 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:48.274315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.286712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5472d82-7ba4-44f8-b884-c0468b98b6b8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.294296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.295036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.295318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.295595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.296249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.297894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.310645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1010/2017] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:48.314765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.328458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8ae21-d8d2-4ee0-accb-253a9b23701f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.345697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.345697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.345697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.345697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] PATH_INFO: `/v3/projects/9027195f9fc74716b5a38828137e3a8d/users/8970213b7027407b94cefb4f2193279e/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.355599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9027195f9fc74716b5a38828137e3a8d, user_id=8970213b7027407b94cefb4f2193279e, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.358313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.374238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1008/2018] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:48 2022] PUT /identity/v3/projects/9027195f9fc74716b5a38828137e3a8d/users/8970213b7027407b94cefb4f2193279e/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:48.378475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.390453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72557-e2a5-4169-9e26-76a91c87e8a8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.462924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.463658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.464064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.464507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.465271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.467515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c971893-ba89-489d-b1c7-8741512367bd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.484922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1011/2019] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:48.489093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.500869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91847b6a-bc21-4453-9dd8-aa470f0ba674 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.506123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.506920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.507206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.507476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] PATH_INFO: `/v3/projects/9027195f9fc74716b5a38828137e3a8d/users/8970213b7027407b94cefb4f2193279e/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.509660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9027195f9fc74716b5a38828137e3a8d, user_id=8970213b7027407b94cefb4f2193279e, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.512452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.524110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1009/2020] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:48 2022] PUT /identity/v3/projects/9027195f9fc74716b5a38828137e3a8d/users/8970213b7027407b94cefb4f2193279e/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:48.534388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.534703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.534981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.565672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] MFA Rules not processed for user `8970213b7027407b94cefb4f2193279e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:48.566913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:48.642172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1012/2021] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:48.653341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:48.655055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:48.720303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8eb5b-671c-4717-8c42-7461cce98792 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:48.725714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:48.726358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:48.726787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:48.726996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:48.738389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:48.739855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:48.743850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:48.752496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1010/2022] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:49.542519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:49.550654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:49.640792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e3f63-9870-405e-9144-5c6aeeb03bc5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:49.646479 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:49.647320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:49.647701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:49.648274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] PATH_INFO: `/v3/users/0e2dcf7120cb4786b59798979738424e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:49.649809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e2dcf7120cb4786b59798979738424e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:49.652039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:49.670089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:49.673030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:49.694887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a22c8-d282-4a41-b597-d5ccd96c25ae service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:49.712179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:49.712899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:49.713390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:49.713726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] PATH_INFO: `/v3/users/c9e29a792d1f4f038f0b975a5f84e869` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:49.716301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e29a792d1f4f038f0b975a5f84e869)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:49.719937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:49.737504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e2dcf7120cb4786b59798979738424e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.755700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e2dcf7120cb4786b59798979738424e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.756560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e2dcf7120cb4786b59798979738424e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.757302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e2dcf7120cb4786b59798979738424e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.772287 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c3b32f96-807d-4a92-b6cd-9d44f28e9681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e2dcf7120cb4786b59798979738424e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.795023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1013/2023] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:49 2022] DELETE /identity/v3/users/0e2dcf7120cb4786b59798979738424e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:49.800693 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:49.801066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:49.801347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:49.821035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e29a792d1f4f038f0b975a5f84e869'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.830322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e29a792d1f4f038f0b975a5f84e869'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.830500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:49.832129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:49.832473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e29a792d1f4f038f0b975a5f84e869'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.833427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e29a792d1f4f038f0b975a5f84e869'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.845927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0aa6f546-966b-403c-96ed-a454e553b2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e29a792d1f4f038f0b975a5f84e869'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:49.875075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1011/2024] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:49 2022] DELETE /identity/v3/users/c9e29a792d1f4f038f0b975a5f84e869 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:49.879375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:49.880484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:49.880955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:49.906502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:49.908028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:49.921044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1014/2025] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:49.932639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:49.935262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:49.975591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:49.992917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1012/2026] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:50.006207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.008036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.034527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a93ba-016e-4c62-b4d3-96fff718e18e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.106961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.108095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.108978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.109464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.130052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.133305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.154179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58487-72ee-41ae-be72-71c2cb0577fb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.159002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.160485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.160918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.161361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.162014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1015/2027] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:50.191501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.193513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec98f6-a0a4-41a1-aaaf-eb8cc7932711 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.207889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1013/2028] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:50.279217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.281126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.303000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f72b97-0972-47fa-838b-4c7fa1ffc560 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.317566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.318563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.319910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.320387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] PATH_INFO: `/v3/users/9b1fa42080164e1ea55bd0dac3f63adb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.324483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1fa42080164e1ea55bd0dac3f63adb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.327894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.406861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1fa42080164e1ea55bd0dac3f63adb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.423481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1fa42080164e1ea55bd0dac3f63adb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.425314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1fa42080164e1ea55bd0dac3f63adb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.426240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1fa42080164e1ea55bd0dac3f63adb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.446126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1fa42080164e1ea55bd0dac3f63adb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.477343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1016/2029] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:50 2022] DELETE /identity/v3/users/9b1fa42080164e1ea55bd0dac3f63adb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:50.482146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.482459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.483291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.555989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:50.556827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.579969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:50.628918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.630527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.666649 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1014/2030] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:50.675513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.679102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.711431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:50.722959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bf22-0997-44fe-954e-f2e7e35dc9c6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.734475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.735293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.736875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.737068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] PATH_INFO: `/v3/users/096dd6902db149369f5963f32d5eeb48` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.738667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=096dd6902db149369f5963f32d5eeb48)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.777222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8f5e-20c1-437d-bb64-6dc1d96e35e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.783434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.785331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.786450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.787139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.802318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.804395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.822898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1015/2031] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:50.824508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.828245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.844991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d25f9-18ca-4772-8dc0-abf7d396111d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:50.850305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:50.850985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:50.851308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:50.851574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] PATH_INFO: `/v3/projects/ad8d57beb105473ea99ee1f3919aa04a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:50.852934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8d57beb105473ea99ee1f3919aa04a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:50.855948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:50.865275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096dd6902db149369f5963f32d5eeb48'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.880914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096dd6902db149369f5963f32d5eeb48'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.881686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096dd6902db149369f5963f32d5eeb48'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.882317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096dd6902db149369f5963f32d5eeb48'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.896982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096dd6902db149369f5963f32d5eeb48'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.926405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1017/2032] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:50 2022] DELETE /identity/v3/users/096dd6902db149369f5963f32d5eeb48 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:50.929737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:50.931075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:50.940499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8d57beb105473ea99ee1f3919aa04a', 'request_id': 'req-5a2f4b2a-8d5d-4123-a416-d8c71a280877'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.954095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8d57beb105473ea99ee1f3919aa04a', 'request_id': 'req-5a2f4b2a-8d5d-4123-a416-d8c71a280877'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.971543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8d57beb105473ea99ee1f3919aa04a', 'request_id': 'req-5a2f4b2a-8d5d-4123-a416-d8c71a280877'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.994512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] The token cache is being invalidate because project ad8d57beb105473ea99ee1f3919aa04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:50.998735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:50.999644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5a2f4b2a-8d5d-4123-a416-d8c71a280877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.001670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1016/2033] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:50 2022] DELETE /identity/v3/projects/ad8d57beb105473ea99ee1f3919aa04a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:51.003019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.003303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.003575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.040995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629f3a-15ab-463b-8f42-452be8f29dcd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:51.048379 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:51.049774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.050203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.050589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] PATH_INFO: `/v3/projects/c366db70acfd403a80504ad1abffaeb4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.051680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:51.052558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c366db70acfd403a80504ad1abffaeb4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:51.052970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.056042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:51.078976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1017/2034] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:51.079902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:51.085113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.108009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94639-7daa-4476-9046-d54220cb6c4c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:51.123011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:51.123824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.124188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.125020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] PATH_INFO: `/v3/users/2101dd0b8a6146ba9887ba93002455ca` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.128039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2101dd0b8a6146ba9887ba93002455ca)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:51.133931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:51.144510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c366db70acfd403a80504ad1abffaeb4', 'request_id': 'req-affb4ad6-432a-4fa5-b730-0369bb0e296a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.168679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c366db70acfd403a80504ad1abffaeb4', 'request_id': 'req-affb4ad6-432a-4fa5-b730-0369bb0e296a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.177968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c366db70acfd403a80504ad1abffaeb4', 'request_id': 'req-affb4ad6-432a-4fa5-b730-0369bb0e296a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.195001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:51.202590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] The token cache is being invalidate because project c366db70acfd403a80504ad1abffaeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:51.203128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.204413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.207631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1018/2035] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:50 2022] DELETE /identity/v3/projects/c366db70acfd403a80504ad1abffaeb4 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:51.211852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:51.213699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.222613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:51.236948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2101dd0b8a6146ba9887ba93002455ca'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.250543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2101dd0b8a6146ba9887ba93002455ca'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.254282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2101dd0b8a6146ba9887ba93002455ca'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.265857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2101dd0b8a6146ba9887ba93002455ca'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.288016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2101dd0b8a6146ba9887ba93002455ca'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.326079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1018/2036] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:51 2022] DELETE /identity/v3/users/2101dd0b8a6146ba9887ba93002455ca => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:51.327266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:51.336819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.338956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ad6-432a-4fa5-b730-0369bb0e296a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:51.398627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:51.399401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.399750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.400102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.402544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.522364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaeda5-7b75-45ac-9a23-437b513a2cbb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:51.538074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:51.538907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.539298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.540461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] PATH_INFO: `/v3/users/8970213b7027407b94cefb4f2193279e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.548096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8970213b7027407b94cefb4f2193279e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:51.548096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:51.564494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:51.571261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:51.582507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1019/2037] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:51.641731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8970213b7027407b94cefb4f2193279e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.656189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8970213b7027407b94cefb4f2193279e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.657383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8970213b7027407b94cefb4f2193279e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.658103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8970213b7027407b94cefb4f2193279e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.677983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8970213b7027407b94cefb4f2193279e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:51.721073 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1019/2038] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:51 2022] DELETE /identity/v3/users/8970213b7027407b94cefb4f2193279e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:51.823691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:51.827497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:51.844544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8009-a3f1-4bfb-abb7-3c3df11263ba service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:51.977274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:51.977990 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:51.978347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:51.978714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] PATH_INFO: `/v3/projects/d228aab06f3c4f798519d2e782320844` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:51.979925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d228aab06f3c4f798519d2e782320844)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:51.982824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:51.998204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:52.047927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.049719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.066019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52237-e91a-4ce1-932f-19d871a6b461 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:52.085302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd228aab06f3c4f798519d2e782320844', 'request_id': 'req-32434cb4-8fb7-4e43-a677-c163ab2aea6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.108467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd228aab06f3c4f798519d2e782320844', 'request_id': 'req-32434cb4-8fb7-4e43-a677-c163ab2aea6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.116983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:52.120182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd228aab06f3c4f798519d2e782320844', 'request_id': 'req-32434cb4-8fb7-4e43-a677-c163ab2aea6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.135840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] The token cache is being invalidate because project d228aab06f3c4f798519d2e782320844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:52.136559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.137405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.139860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1020/2039] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:51 2022] DELETE /identity/v3/projects/d228aab06f3c4f798519d2e782320844 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:52.147126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.148808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.192703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:52.193368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.193754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.193907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] PATH_INFO: `/v3/projects/e2956355677a4f73b443955000ad4899` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.195891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2956355677a4f73b443955000ad4899)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:52.197792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:52.236028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434cb4-8fb7-4e43-a677-c163ab2aea6a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:52.258441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:52.259104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.259388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.259665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] PATH_INFO: `/v3/users/249adcc220fd4debb3dd537ccbebb957` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.260867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=249adcc220fd4debb3dd537ccbebb957)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:52.263395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:52.274535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2956355677a4f73b443955000ad4899', 'request_id': 'req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.283456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2956355677a4f73b443955000ad4899', 'request_id': 'req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.298358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2956355677a4f73b443955000ad4899', 'request_id': 'req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.320600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] The token cache is being invalidate because project e2956355677a4f73b443955000ad4899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:52.320997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.323264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.327092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1020/2040] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/projects/e2956355677a4f73b443955000ad4899 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:52.327839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.329289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.336403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249adcc220fd4debb3dd537ccbebb957'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.346239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249adcc220fd4debb3dd537ccbebb957'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.347692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249adcc220fd4debb3dd537ccbebb957'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.349074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249adcc220fd4debb3dd537ccbebb957'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.360780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9fed279c-8062-4f45-a976-e624196a1554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249adcc220fd4debb3dd537ccbebb957'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.393795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1021/2041] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/users/249adcc220fd4debb3dd537ccbebb957 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:52.393795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.393795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.393795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.444921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9256d2-c691-47a5-8479-f2cd6ffb4b81 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:52.472084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:52.483967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.513616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:52.574972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:52.576305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.576590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.576866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] PATH_INFO: `/v3/projects/0f07861ca014450280f968c4cd49cacc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.578161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f07861ca014450280f968c4cd49cacc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:52.580173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:52.584722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:52.611332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1022/2042] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:52.612592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.625413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.643610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e779486-52df-4861-b0d1-1b466c520bc4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:52.652321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:52.654159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.654514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.654800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] PATH_INFO: `/v3/projects/9027195f9fc74716b5a38828137e3a8d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.656081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9027195f9fc74716b5a38828137e3a8d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:52.658162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:52.691753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f07861ca014450280f968c4cd49cacc', 'request_id': 'req-ef2e409a-f94c-4e23-9447-c694355d9507'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.700881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f07861ca014450280f968c4cd49cacc', 'request_id': 'req-ef2e409a-f94c-4e23-9447-c694355d9507'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.713306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f07861ca014450280f968c4cd49cacc', 'request_id': 'req-ef2e409a-f94c-4e23-9447-c694355d9507'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.732905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9027195f9fc74716b5a38828137e3a8d', 'request_id': 'req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.738135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] The token cache is being invalidate because project 0f07861ca014450280f968c4cd49cacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:52.738518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.739174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef2e409a-f94c-4e23-9447-c694355d9507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.744547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1021/2043] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/projects/0f07861ca014450280f968c4cd49cacc => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:52.746084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9027195f9fc74716b5a38828137e3a8d', 'request_id': 'req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.749250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a878-2b06-4042-a182-d6501ef0cd2a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.749534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a878-2b06-4042-a182-d6501ef0cd2a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.750848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a878-2b06-4042-a182-d6501ef0cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.755048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9027195f9fc74716b5a38828137e3a8d', 'request_id': 'req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.770609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] The token cache is being invalidate because project 9027195f9fc74716b5a38828137e3a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:52.771052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.775991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:52.785438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1023/2044] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/projects/9027195f9fc74716b5a38828137e3a8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:52.787461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.790406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.790955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9ff7a878-2b06-4042-a182-d6501ef0cd2a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:52.792368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9ff7a878-2b06-4042-a182-d6501ef0cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.891907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94e8-1d8d-4f34-9d6b-e13cd23ca1bf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:52.897118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:52.897910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.898190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.898490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] PATH_INFO: `/v3/users/69ee174a9dd94a9387d56d0f0aeb9bb3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.899842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ee174a9dd94a9387d56d0f0aeb9bb3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:52.901794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:52.921759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1022/2045] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:52.923345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:52.923684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:52.924110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:52.966171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:52.968493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.979690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1023/2046] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:52.980665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:52.982339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:52.991884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:52.995573 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ee174a9dd94a9387d56d0f0aeb9bb3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.001034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bb377-de40-4337-a3c9-77c21d789a37 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.006103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ee174a9dd94a9387d56d0f0aeb9bb3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.006933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.007260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ee174a9dd94a9387d56d0f0aeb9bb3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.007600 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.007969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.008239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] PATH_INFO: `/v3/projects/e183ffbca4a04d2ab48f53332d3cd725` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.008360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ee174a9dd94a9387d56d0f0aeb9bb3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.009976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e183ffbca4a04d2ab48f53332d3cd725)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.012045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.020525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ee174a9dd94a9387d56d0f0aeb9bb3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.057635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1024/2047] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/users/69ee174a9dd94a9387d56d0f0aeb9bb3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:53.062538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.065854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.106906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e183ffbca4a04d2ab48f53332d3cd725', 'request_id': 'req-7d9e2ada-c450-435b-a343-c71d4026ad8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.115047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e183ffbca4a04d2ab48f53332d3cd725', 'request_id': 'req-7d9e2ada-c450-435b-a343-c71d4026ad8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.126822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e183ffbca4a04d2ab48f53332d3cd725', 'request_id': 'req-7d9e2ada-c450-435b-a343-c71d4026ad8b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.145513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] The token cache is being invalidate because project e183ffbca4a04d2ab48f53332d3cd725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:53.145757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.147402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d9e2ada-c450-435b-a343-c71d4026ad8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.154093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1024/2048] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:52 2022] DELETE /identity/v3/projects/e183ffbca4a04d2ab48f53332d3cd725 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:53.154093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581bd64-d7ec-4d85-ade9-a4a70e780d27 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.154093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581bd64-d7ec-4d85-ade9-a4a70e780d27 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.154093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581bd64-d7ec-4d85-ade9-a4a70e780d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.175444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9581bd64-d7ec-4d85-ade9-a4a70e780d27 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:53.177903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9581bd64-d7ec-4d85-ade9-a4a70e780d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.181339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655897f-4eca-4ada-8c3e-da595a063007 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.187937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:53.270071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1025/2049] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:53.274561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.274854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.275136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.280726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.281345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.281628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.281909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.282603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:53.283442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.284540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.304825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1025/2050] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:53.304993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:53.305093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.305181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.313246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1026/2051] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:53.317416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b283e78-4af0-44b1-a310-5b69fecf4872 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.320798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.322194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.323303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.323913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.324298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.324606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.325882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:53.326194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.327992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146e687-4c72-47e1-9dea-3089567855de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.337375 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b866a-417b-49d2-adca-065735675e79 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.343466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.344076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.344358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.344632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.345500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.345609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1026/2052] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:53.347234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.348182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.349921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.350211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.388551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1027/2053] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:53.388551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.388849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:53.388849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.520686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466eda-99ab-401a-ae64-94c1550266a5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.525448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:53.527059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.528184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.528736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.529206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.530143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.532932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.569406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1027/2054] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:53.572877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1028/2055] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:53.573022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.573124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.574510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.584303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b90e-fda6-49d7-afad-03e300260cca None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.650659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.651239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdfbd9-0222-4b0f-9653-b0e5864f6b56 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.651384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.651525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.651790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.652419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.654581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.656265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.656910 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.657189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.657464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] PATH_INFO: `/v3/projects/a6e50051df0c4900842b834c0ef0c00b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.658629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e50051df0c4900842b834c0ef0c00b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.660557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.669461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:53.678335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1028/2056] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:53.679002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.689524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d0522-9895-4ce8-8609-ebc87fd65075 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.695670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.696355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.696700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.696973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.697669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:53.697959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.699644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.712849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1029/2057] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:53.713640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.714688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e50051df0c4900842b834c0ef0c00b', 'request_id': 'req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.721080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e50051df0c4900842b834c0ef0c00b', 'request_id': 'req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.726178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8e5-f51c-467d-a349-3a0a57d927cb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.729450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e50051df0c4900842b834c0ef0c00b', 'request_id': 'req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.742564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] The token cache is being invalidate because project a6e50051df0c4900842b834c0ef0c00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:53.742786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.743453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.745176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1029/2058] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:53 2022] DELETE /identity/v3/projects/a6e50051df0c4900842b834c0ef0c00b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:53.745816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.747914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.793348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.794497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.794771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.795038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.795663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.797555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.821114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cf86c-3cd6-435b-90b6-6e0be31e0b98 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.822878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1030/2059] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:53.823520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.825295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.829478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.830070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.830354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.830664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.831309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.832925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.836708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae640-f05d-4d2d-95e5-dc6e1b3009b0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.841684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.842309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.842619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.842886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.843506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.843826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1030/2060] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:53.844624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.845553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.848414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.863228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704ca88-d8fb-4267-a0f6-92fdd2857501 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.868812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.869438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.869746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.870009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] PATH_INFO: `/v3/users/89eee876aacd400f8d9ac389ce170396` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.871188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89eee876aacd400f8d9ac389ce170396)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.873638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.877326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1031/2061] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:53.878283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.882191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.934006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:53.938372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89eee876aacd400f8d9ac389ce170396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.947033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89eee876aacd400f8d9ac389ce170396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.947763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89eee876aacd400f8d9ac389ce170396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.948394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89eee876aacd400f8d9ac389ce170396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.957664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89eee876aacd400f8d9ac389ce170396'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:53.966443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabbef6-def5-42fc-94fc-5fb1f3f4bbb2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:53.974387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:53.975588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:53.975939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:53.976232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:53.977888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:53.980273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:53.984727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1031/2062] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:53 2022] DELETE /identity/v3/users/89eee876aacd400f8d9ac389ce170396 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:53.985723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.994749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:53.998849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1032/2063] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:53.998849 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.000968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.016147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:54.037068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a37cb-c0a5-49e8-a445-77704281dfc5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.158563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4486b-1eea-42d5-8696-d636a3d7d698 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.174841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.176470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.177390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.177675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.178066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.179008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.182865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.184531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.187961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.188879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.190444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.193824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.228189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1032/2064] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:54.228189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.229031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1033/2065] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:53 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:54.229031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.229196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.245556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8db1-60d8-487f-8f4b-08fa98b1bf16 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.252598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.253184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.253466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.253741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] PATH_INFO: `/v3/projects/ad21175a38b44ef182334c2a30c7afeb/users/ecb1fc0cc0974567a52a53f23926fe46/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.301260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a9f5-25ec-47f0-86ab-5a2c6ef73ae1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.307678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad21175a38b44ef182334c2a30c7afeb, user_id=ecb1fc0cc0974567a52a53f23926fe46, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.313714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.314363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.314621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.314897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.315540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.317160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.321357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.346731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1034/2066] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:54 2022] PUT /identity/v3/projects/ad21175a38b44ef182334c2a30c7afeb/users/ecb1fc0cc0974567a52a53f23926fe46/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:54.347715 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.349262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1033/2067] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:54.364997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.384414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635813aa-ceb8-45b9-89be-8e71ea7b76d6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.524686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6ca95-afeb-4003-8ac1-8b98de2bfaed admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.534672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.536387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.536951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.537534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] PATH_INFO: `/v3/projects/de1a6051dcf943b79624ac54a9cf6855/users/492bbe43ecc343cf94b5b77ed5885b2c/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.546893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.547541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.547818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.548086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.548894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.550550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a87a0f-dcda-4575-9c17-63b1a06dd8d7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.560573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1a6051dcf943b79624ac54a9cf6855, user_id=492bbe43ecc343cf94b5b77ed5885b2c, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.563809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.572853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1035/2068] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:54.574140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.575606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.575606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.591657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1034/2069] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:54 2022] PUT /identity/v3/projects/de1a6051dcf943b79624ac54a9cf6855/users/492bbe43ecc343cf94b5b77ed5885b2c/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:54.592655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.623296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] MFA Rules not processed for user `ecb1fc0cc0974567a52a53f23926fe46`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:54.624579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.648898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:54.698920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef8e0-78b0-486a-8b2e-5188073491c5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.706002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.707713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.708438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.708835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.709559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.712136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807b605-f894-4752-a8b6-e0f421c21f42 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.729732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1035/2070] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:54.732759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.733130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.733410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.744337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1036/2071] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:54.750248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.764932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d398129-a5b2-41ae-8efa-8dc2880bed13 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.769754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:54.769995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.770931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.771245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.771660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] PATH_INFO: `/v3/projects/eb261d24ad9542c081d5f1edace5f546/users/8463e01dc6a94a068e037ad2c4f157c0/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.784645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] MFA Rules not processed for user `492bbe43ecc343cf94b5b77ed5885b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:54.785866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.791759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb261d24ad9542c081d5f1edace5f546, user_id=8463e01dc6a94a068e037ad2c4f157c0, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.796697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.820514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1037/2072] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:54 2022] PUT /identity/v3/projects/eb261d24ad9542c081d5f1edace5f546/users/8463e01dc6a94a068e037ad2c4f157c0/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:54.821709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.882273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1036/2073] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:54.884211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.885630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.901704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c652b-365f-45c7-b3ea-d18a1465a2cd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.929184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ee2b4-5bf3-4e2c-8f77-9392c2fd685f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.934286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.934948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.935198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.935472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.936108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.938019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cab64-44a4-49ff-801b-4977a9e9ba2a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.968415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1038/2074] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:54 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:54.969584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.969873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.970148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.980355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.981042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.982463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.982746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] PATH_INFO: `/v3/projects/3c2cd65e62324421abd1cac27aa850e7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.983865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2cd65e62324421abd1cac27aa850e7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.985854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.999260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] MFA Rules not processed for user `8463e01dc6a94a068e037ad2c4f157c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:55.000509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.032863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2cd65e62324421abd1cac27aa850e7', 'request_id': 'req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.038587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2cd65e62324421abd1cac27aa850e7', 'request_id': 'req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.046795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2cd65e62324421abd1cac27aa850e7', 'request_id': 'req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.058701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] The token cache is being invalidate because project 3c2cd65e62324421abd1cac27aa850e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:55.058989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.059671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.070456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1037/2075] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:54 2022] DELETE /identity/v3/projects/3c2cd65e62324421abd1cac27aa850e7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:55.070456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.070456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.077111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1039/2076] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:55.078773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.097860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.116962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.164606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f172-9eaf-4bf6-9406-7af341096c93 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.173093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6ad4e-2cda-429f-8056-f7b4b4b31ab3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.178186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.179135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.179367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.179636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.181645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.239149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.239677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.239958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.240234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.240880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.242875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.269504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1040/2077] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/users => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:55.271264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.271889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.273474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.273963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.285119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1038/2078] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:55.287907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.289231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.302939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591fde-94fd-499b-a0c0-6d1cad6c496d service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.313489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.314203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.314515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.314795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.315446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.317041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.329343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1039/2079] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:55.333798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.336595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e30fa-71a8-41ae-bbe6-9ea069adbde9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.341697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.342237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.342669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.342947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.344800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.347507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ab96a-c641-4f14-b056-47276736f9d8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.352204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.352991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.353269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.353544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] PATH_INFO: `/v3/projects/e6fe65113b2d4499ac580aa8d96469d7/users/655956e76bf2416cb901d582814fb80b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.368713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6fe65113b2d4499ac580aa8d96469d7, user_id=655956e76bf2416cb901d582814fb80b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.371370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.386685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1040/2080] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:55 2022] PUT /identity/v3/projects/e6fe65113b2d4499ac580aa8d96469d7/users/655956e76bf2416cb901d582814fb80b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:55.391101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.411188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.412634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.428341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1041/2081] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:55.473825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5a42-7d34-4cd7-9236-339f3a003878 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.480492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.481551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.481937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.482281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.483049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.484897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888346-7fb7-48ed-931d-ac2cf6d5e959 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.498472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1041/2082] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:55.503433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.511108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.515846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b343e-fa45-4b38-a460-93e8e6e724cc service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.524286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.525240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.525614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.526150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] PATH_INFO: `/v3/projects/e6fe65113b2d4499ac580aa8d96469d7/users/655956e76bf2416cb901d582814fb80b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.529906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6fe65113b2d4499ac580aa8d96469d7, user_id=655956e76bf2416cb901d582814fb80b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.532640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.543902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1042/2083] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:55 2022] PUT /identity/v3/projects/e6fe65113b2d4499ac580aa8d96469d7/users/655956e76bf2416cb901d582814fb80b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:55.554313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.554638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.554921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.588220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] MFA Rules not processed for user `655956e76bf2416cb901d582814fb80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:55.589450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.603903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.670781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1042/2084] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:55.804748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.875453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12ddc-6b79-42cd-abef-30ea2d87b871 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.879858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.880698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.881740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.882083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.894380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.896125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.910334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1043/2085] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:56.140755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.150595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b7713-a7cd-4879-9bbc-509ace5f101e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.154092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.154889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.155013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.155283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.156989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:56.223177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.224644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.234547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1043/2086] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:59.907971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.910264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:59.991284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253632d-e9f8-4af6-bffa-aaabdbde3eb5 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.996409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.997270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.997556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.997830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] PATH_INFO: `/v3/users/b410ecaf96f44dd2bc9c89f82a0099bc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.998999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b410ecaf96f44dd2bc9c89f82a0099bc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.000957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.048063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b410ecaf96f44dd2bc9c89f82a0099bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.055013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b410ecaf96f44dd2bc9c89f82a0099bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.055667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b410ecaf96f44dd2bc9c89f82a0099bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.056255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b410ecaf96f44dd2bc9c89f82a0099bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.064392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b410ecaf96f44dd2bc9c89f82a0099bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.085216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1044/2087] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:59 2022] DELETE /identity/v3/users/b410ecaf96f44dd2bc9c89f82a0099bc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:00.265507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.277420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecc4a0-9255-4568-b001-2e1f899a423c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.357892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.358602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.358886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.359199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] PATH_INFO: `/v3/projects/b7fc79cd5d144cec991c96c4e72ec0a8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.360389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7fc79cd5d144cec991c96c4e72ec0a8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.363337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.415166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7fc79cd5d144cec991c96c4e72ec0a8', 'request_id': 'req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.416439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:00.421899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7fc79cd5d144cec991c96c4e72ec0a8', 'request_id': 'req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.430641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7fc79cd5d144cec991c96c4e72ec0a8', 'request_id': 'req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.445048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] The token cache is being invalidate because project b7fc79cd5d144cec991c96c4e72ec0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:00.445290 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.445965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.447527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1044/2088] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:00 2022] DELETE /identity/v3/projects/b7fc79cd5d144cec991c96c4e72ec0a8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:00.452372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.453818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.467469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8be4a-849c-4452-b055-5ac48e9f206c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.476900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:00.539977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.540711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.541039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.541339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] PATH_INFO: `/v3/users/e8fbf91c5a4d4ac9a742816831a353a7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.542618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fbf91c5a4d4ac9a742816831a353a7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.545158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.602739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fbf91c5a4d4ac9a742816831a353a7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.610601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fbf91c5a4d4ac9a742816831a353a7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.611388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fbf91c5a4d4ac9a742816831a353a7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.612059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fbf91c5a4d4ac9a742816831a353a7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.622383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c67310f7-eaec-459e-9b62-b35c0be1bde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fbf91c5a4d4ac9a742816831a353a7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.641452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1045/2089] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:00 2022] DELETE /identity/v3/users/e8fbf91c5a4d4ac9a742816831a353a7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:00.842607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.846453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.885692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b258b6-c4da-4dd0-9d40-47feb269bf8f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.965012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.966301 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.966708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.967079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] PATH_INFO: `/v3/projects/ff09cf986a604e03bc565a7df6a4e002` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.968291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff09cf986a604e03bc565a7df6a4e002)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.970207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.017489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff09cf986a604e03bc565a7df6a4e002', 'request_id': 'req-3bef1241-3256-4e69-a8ea-f37b7f835e95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:01.022956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff09cf986a604e03bc565a7df6a4e002', 'request_id': 'req-3bef1241-3256-4e69-a8ea-f37b7f835e95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:01.031962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff09cf986a604e03bc565a7df6a4e002', 'request_id': 'req-3bef1241-3256-4e69-a8ea-f37b7f835e95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:01.050076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] The token cache is being invalidate because project ff09cf986a604e03bc565a7df6a4e002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:01.050374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:01.051062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3bef1241-3256-4e69-a8ea-f37b7f835e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:01.052540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1045/2090] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:00 2022] DELETE /identity/v3/projects/ff09cf986a604e03bc565a7df6a4e002 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:01.057381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.057868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.057973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.077068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:01.078244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:01.134966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1046/2091] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:01.143206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.143487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.143756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.163680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:01.164911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:01.173499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1046/2092] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:01.177698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.187684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9da54f-1f38-42fe-a50b-341a8e442a5b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.192346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.192940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.193222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.193492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.194201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:01.194545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.196251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.208554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1047/2093] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:01.212903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.222783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf951e10-faab-44b0-b53f-c405ad17758c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.228130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.228130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.228377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.228629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.229281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.231223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.266012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1047/2094] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:01.267301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.278377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4e12-1dcf-4287-9ae4-d1937dcfe713 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.296195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:01.339844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.340940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.341206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.341481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.342312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.344156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.366669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1048/2095] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:01.370779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.380906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb90f-d610-4a0a-8f9f-32669fcb022b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.385464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.386057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.386330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.386637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.387996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.389576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.399927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1048/2096] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:01.404291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.417222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b5099-7634-43f7-9f4e-e65016294c44 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.420712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.421460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.421748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.422026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.422713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.424317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1973ed-6c68-4a88-a06b-6867adb8539a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.438871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1049/2097] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:01.440310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.451314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacaac3-8d14-4529-b6ab-21f109f5c339 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.455938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.456591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.456868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.457135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] PATH_INFO: `/v3/projects/d6b541ed034347bbb5393caafac101e7/users/09f37b3a2cb548788794641999068d65/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.472406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b541ed034347bbb5393caafac101e7, user_id=09f37b3a2cb548788794641999068d65, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.475085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.487737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1049/2098] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:01 2022] PUT /identity/v3/projects/d6b541ed034347bbb5393caafac101e7/users/09f37b3a2cb548788794641999068d65/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:01.498978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.499265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.499543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.529198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] MFA Rules not processed for user `09f37b3a2cb548788794641999068d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:01.530321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:01.595381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1050/2099] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:01.600021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.605511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:01.665159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe92612-ca9a-4237-b7ed-2f142c76eaae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.669978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.670608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.670942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.671241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.671956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.674043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.702021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1050/2100] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:01.706189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.717153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060129b9-d9e1-4637-8817-d4635e414f02 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.776579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.777186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.777466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.777735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.778366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.780390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.804728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1051/2101] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:01 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:01.809579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.819904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a2cd-a5a9-4c65-8c19-c448c503b2a8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.825222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.825843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.826120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.826488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.827168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.828818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.839899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1051/2102] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:01.844342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.854279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ca89-cf2c-4721-b1d5-eaece368d1e9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.858982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.859629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.859909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.860187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] PATH_INFO: `/v3/projects/eb755534312141f7b62968d71a337a5f/users/4b32dc8f833d414f8167aa2bbc551102/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.875879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb755534312141f7b62968d71a337a5f, user_id=4b32dc8f833d414f8167aa2bbc551102, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.878642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.881057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:01.894384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1052/2103] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:01 2022] PUT /identity/v3/projects/eb755534312141f7b62968d71a337a5f/users/4b32dc8f833d414f8167aa2bbc551102/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:01.898596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.909356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394742c-ab2e-4198-9277-c274ec529821 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.984289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.985200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.985477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.986185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.986468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.987996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ce44c-5c92-458b-9f5a-c10c6b0c3c2b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:02.001862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1052/2104] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:02.003177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:02.015159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcaca-bc8b-4c63-9ccc-9008ca9a6d95 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:02.020232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:02.020960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:02.021328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:02.021663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] PATH_INFO: `/v3/projects/eb755534312141f7b62968d71a337a5f/users/4b32dc8f833d414f8167aa2bbc551102/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:02.023637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb755534312141f7b62968d71a337a5f, user_id=4b32dc8f833d414f8167aa2bbc551102, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:02.026310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:02.037523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1053/2105] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:02 2022] PUT /identity/v3/projects/eb755534312141f7b62968d71a337a5f/users/4b32dc8f833d414f8167aa2bbc551102/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:02.048759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:02.049045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:02.049323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:02.081394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] MFA Rules not processed for user `4b32dc8f833d414f8167aa2bbc551102`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:02.082707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:02.164243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:02.169577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1053/2106] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:02.181075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:02.182821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:02.239244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875241-23e2-4e0e-b334-5c1018d2743c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:02.242871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:02.243469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:02.243752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:02.244029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:02.300769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:02.302229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:02.312784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1054/2107] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:11.452561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.454832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.583281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b0b6a-9794-47ce-a4e3-d399ca43108f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.595367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.596977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.599226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.599526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] PATH_INFO: `/v3/users/8463e01dc6a94a068e037ad2c4f157c0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.601119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8463e01dc6a94a068e037ad2c4f157c0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.603272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.605036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.608260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.643983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b326e-398f-426d-b0dd-a40c31146f17 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.650407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.650708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.650795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.651202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] PATH_INFO: `/v3/users/ecb1fc0cc0974567a52a53f23926fe46` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.652365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb1fc0cc0974567a52a53f23926fe46)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.654895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.674185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8463e01dc6a94a068e037ad2c4f157c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.684277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8463e01dc6a94a068e037ad2c4f157c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.685031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8463e01dc6a94a068e037ad2c4f157c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.686372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8463e01dc6a94a068e037ad2c4f157c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.694914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2eaa1d17-1151-4cf9-ab6d-0f41a00ebcb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8463e01dc6a94a068e037ad2c4f157c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.709611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb1fc0cc0974567a52a53f23926fe46'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.714114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1054/2108] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:11 2022] DELETE /identity/v3/users/8463e01dc6a94a068e037ad2c4f157c0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:11.717457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb1fc0cc0974567a52a53f23926fe46'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.718149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb1fc0cc0974567a52a53f23926fe46'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.718784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb1fc0cc0974567a52a53f23926fe46'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.718898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.719093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.719402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.726718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5339fe4c-0019-45e4-ac56-3e9d49e40f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb1fc0cc0974567a52a53f23926fe46'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.741695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:11.742930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.745098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1055/2109] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:11 2022] DELETE /identity/v3/users/ecb1fc0cc0974567a52a53f23926fe46 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:11.750378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.750694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.750965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.763745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:11.770773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:11.772485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.803567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1055/2110] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:11.814342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.816355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.841435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1056/2111] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:11.843616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:11.852353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.863633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.883536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abae5f-87e0-455c-b2db-21b4f03b2397 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.887497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.888128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.888390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.888699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.898983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.900511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.911828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1056/2112] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:11.928837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926234c-c662-4bcc-8ce6-302524e07d50 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.932823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.933699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.933975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.934244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.944580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.946301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.957383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1057/2113] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:12.206685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.208712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.231404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c69bc-61eb-45a8-ab2c-55fad30120bc service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.239025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.239677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.239957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.240268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] PATH_INFO: `/v3/projects/eb261d24ad9542c081d5f1edace5f546` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.241452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb261d24ad9542c081d5f1edace5f546)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.249186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.249370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.251493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.269444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca410ec-ec65-42aa-9b79-246e05163dbf service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.274307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.274966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.275246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.275515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] PATH_INFO: `/v3/projects/ad21175a38b44ef182334c2a30c7afeb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.276793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad21175a38b44ef182334c2a30c7afeb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.278767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.310062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb261d24ad9542c081d5f1edace5f546', 'request_id': 'req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.316950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb261d24ad9542c081d5f1edace5f546', 'request_id': 'req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.324307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb261d24ad9542c081d5f1edace5f546', 'request_id': 'req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.335656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad21175a38b44ef182334c2a30c7afeb', 'request_id': 'req-d693f44f-7a39-4e61-8c65-ecb4dc532077'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.336459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] The token cache is being invalidate because project eb261d24ad9542c081d5f1edace5f546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:12.336810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.337469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-61b7da13-4bea-42e3-8cbc-2a95ee3cb70f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.339214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1057/2114] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:12 2022] DELETE /identity/v3/projects/eb261d24ad9542c081d5f1edace5f546 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:12.344609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b3dc-0151-4fa6-9e67-64da8d0adf1b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.345103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b3dc-0151-4fa6-9e67-64da8d0adf1b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.345287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b3dc-0151-4fa6-9e67-64da8d0adf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.349481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad21175a38b44ef182334c2a30c7afeb', 'request_id': 'req-d693f44f-7a39-4e61-8c65-ecb4dc532077'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.358320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad21175a38b44ef182334c2a30c7afeb', 'request_id': 'req-d693f44f-7a39-4e61-8c65-ecb4dc532077'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.366718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-39a0b3dc-0151-4fa6-9e67-64da8d0adf1b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:12.368006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-39a0b3dc-0151-4fa6-9e67-64da8d0adf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.375202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] The token cache is being invalidate because project ad21175a38b44ef182334c2a30c7afeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:12.375486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.376129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.377972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1058/2115] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:12 2022] DELETE /identity/v3/projects/ad21175a38b44ef182334c2a30c7afeb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:12.381669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.383182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.441060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1058/2116] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:12.452103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.452401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.452660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.459157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:12.491823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:12.493377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.503673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1059/2117] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:12.508245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.516434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f44f-7a39-4e61-8c65-ecb4dc532077 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.518057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0051a57-ee16-4ae9-b035-e7075d35756d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.521374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.522021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.522305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.522617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] PATH_INFO: `/v3/users/655956e76bf2416cb901d582814fb80b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.523652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.523883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655956e76bf2416cb901d582814fb80b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.524388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.524740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.525025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.525776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:12.526146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.526267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.528039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.543238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1060/2118] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:12.550438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.565121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f101c2-b9a8-411e-a8bd-ee3a7898190d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.573695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.574751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.577664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.577966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.581327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.583308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.595453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655956e76bf2416cb901d582814fb80b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.619267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655956e76bf2416cb901d582814fb80b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.620148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655956e76bf2416cb901d582814fb80b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.620904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655956e76bf2416cb901d582814fb80b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.633712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655956e76bf2416cb901d582814fb80b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:12.640166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1061/2119] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:12 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:12.643365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.656836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:12.665923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1059/2120] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:12 2022] DELETE /identity/v3/users/655956e76bf2416cb901d582814fb80b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:12.672525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:12.778658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d82b0-c4d2-4a13-ac94-1d1f914d9e19 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.873894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.874625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.874909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.875184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.876084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.878676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.916123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1062/2121] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:12 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:12.917880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.946517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851dd86-d9c2-4084-856d-d7621ac6fb61 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.946517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.946517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.946517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.946998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.946998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.946998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:12.962087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1060/2122] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:12.966032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:12.976712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d719d-b6b3-4d86-a5d4-8fe237b118d5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:12.983853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:12.984525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:12.984811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:12.985116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:12.985777 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:12.988086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481d56d-1d82-4075-baeb-c80524421b5f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.002459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1063/2123] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:13.004542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.018996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729d908-d111-4992-bc43-9081246b3bb6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.024642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.025201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.025485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.025762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] PATH_INFO: `/v3/projects/657ee920907c40d69947f3d2866b92e3/users/46cbcc750dd0495883f2025a804bb0f5/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.071199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=657ee920907c40d69947f3d2866b92e3, user_id=46cbcc750dd0495883f2025a804bb0f5, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.074334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.096813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1061/2124] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:13 2022] PUT /identity/v3/projects/657ee920907c40d69947f3d2866b92e3/users/46cbcc750dd0495883f2025a804bb0f5/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:13.108529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.108826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.109111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.148182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] MFA Rules not processed for user `46cbcc750dd0495883f2025a804bb0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.149421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.172280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.175004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.231973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1064/2125] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:13.244327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.246180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.260162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0a275-2792-4551-88c6-dc7fb890ac7c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.265581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.266219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.266526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.267335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] PATH_INFO: `/v3/projects/e6fe65113b2d4499ac580aa8d96469d7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.268089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6fe65113b2d4499ac580aa8d96469d7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.270005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.281269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:13.294109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:13.324309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b088396-9a65-4008-b5d0-399b6b881561 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.324794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6fe65113b2d4499ac580aa8d96469d7', 'request_id': 'req-ab2e2598-b25a-4970-97ad-64a308dbb40a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.333965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6fe65113b2d4499ac580aa8d96469d7', 'request_id': 'req-ab2e2598-b25a-4970-97ad-64a308dbb40a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.341034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6fe65113b2d4499ac580aa8d96469d7', 'request_id': 'req-ab2e2598-b25a-4970-97ad-64a308dbb40a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.356952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] The token cache is being invalidate because project e6fe65113b2d4499ac580aa8d96469d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:13.358020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.358783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ab2e2598-b25a-4970-97ad-64a308dbb40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.360478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1062/2126] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:13 2022] DELETE /identity/v3/projects/e6fe65113b2d4499ac580aa8d96469d7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:13.365129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.365406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.365679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.377980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.378791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.379072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.379345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.381728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.385431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.386640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.442203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.443838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa640e-0c88-45dc-b4bb-337166115c5a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.461065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1065/2127] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:13.461683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1063/2128] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:13.463069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.463425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.463843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.485328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.486603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.497904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1066/2129] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:13.499879 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.510504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd646730-ceb7-4678-988a-f3897811eb85 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.515198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.515801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.516090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.516356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.517009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:13.517317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.519189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.536284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1064/2130] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:13.538013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.548703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32160e51-ba30-402e-8c9f-fcf8375e4a97 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.555269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.556199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.556683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.557155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.557886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.560129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.592783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1067/2131] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:13.594181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.605183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8da9-33e2-486c-9952-5e908c1423b7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.677981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.678637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.678895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.679168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.680376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.681882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.706701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1065/2132] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:13.711731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.725554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774aba-6fbf-456e-8bb7-0cae1c247d27 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.738783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.739546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.739936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.740321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.741132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.743137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.756912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1068/2133] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:13.760673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.772890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3925372-ce13-4ec7-8865-0236a461b937 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.781544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.782585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.782988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.783302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.784411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.787293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9619e-f214-40a7-bcad-85063b07c029 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.805519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1066/2134] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:13.811734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.823906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28920eed-79a2-49bf-a954-11ea31f49fb9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.831031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.832293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.832293 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.832614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] PATH_INFO: `/v3/projects/0da1d4f8e77c40f3b89014e3e440320d/users/2392d440b8ff4f95b5235d231e9282df/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.853564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da1d4f8e77c40f3b89014e3e440320d, user_id=2392d440b8ff4f95b5235d231e9282df, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.856742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.880165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1069/2135] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:13 2022] PUT /identity/v3/projects/0da1d4f8e77c40f3b89014e3e440320d/users/2392d440b8ff4f95b5235d231e9282df/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:13.891130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.891416 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.891712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.923435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:13.926909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] MFA Rules not processed for user `2392d440b8ff4f95b5235d231e9282df`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.928767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.021237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1067/2136] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:14.025816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.066830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:14.107978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960561-8eaa-4990-b13f-a4a063001444 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.112892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.113576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.113919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.114264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.115189 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.117065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.168168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.175747 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1070/2137] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:14 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:14.183271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d0705-5e1f-46fc-a934-1d88edd23570 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.295088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.295088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.295088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.295088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.295505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.295505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.326837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.330510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1068/2138] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:14 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:14.335875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.338214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.338596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f69e6-956a-489e-b914-c40801849bcb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.343932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.344638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.345031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.345415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.346255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.348253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.362582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1071/2139] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:14.363739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.374531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9788fe8-17f6-4d2a-8dfa-e40daeeaae40 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.380715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.381387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.381663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.381935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] PATH_INFO: `/v3/projects/94dce9d809fc4947b015a037981779ca/users/2922ccc6340749bdb472e00d663dd95b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.400407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dce9d809fc4947b015a037981779ca, user_id=2922ccc6340749bdb472e00d663dd95b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.403243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.415090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e286d-1a0f-4518-8fa7-b388e4e28ecf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.423883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.424667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.425072 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.425460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.427444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.432465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1072/2140] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:14 2022] PUT /identity/v3/projects/94dce9d809fc4947b015a037981779ca/users/2922ccc6340749bdb472e00d663dd95b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:14.434449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.492353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:14.504963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd640899-13d2-4a9a-9946-ea236ffb462f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.511662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.512276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.513157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.513504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.514896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.515738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.516624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588a4ef-d210-40af-bc36-acd9d0e538e6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.517399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.527699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1069/2141] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:14.529848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1073/2142] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:14.534089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.545696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83751fcf-da74-4c7c-8b4b-c402e11a07a9 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.551444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.552107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.552382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.552668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] PATH_INFO: `/v3/projects/94dce9d809fc4947b015a037981779ca/users/2922ccc6340749bdb472e00d663dd95b/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.554757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dce9d809fc4947b015a037981779ca, user_id=2922ccc6340749bdb472e00d663dd95b, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.557508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.570903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1070/2143] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:14 2022] PUT /identity/v3/projects/94dce9d809fc4947b015a037981779ca/users/2922ccc6340749bdb472e00d663dd95b/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:14.583159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.583885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.584335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.616950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] MFA Rules not processed for user `2922ccc6340749bdb472e00d663dd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:14.618289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.694784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:14.707102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1074/2144] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:14.716757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.718867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.813657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132929fd-35a7-4fd8-ba14-9c194f6f2118 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.819009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.820054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.820413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.820793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.835260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.837363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efb922-3491-4499-9457-1edc40b4fff1 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.853132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1071/2145] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:15.839165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:15.910155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dafe9-af67-48d0-a2fe-b8476ce39985 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:15.915222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:15.915840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:15.916111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:15.916440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] PATH_INFO: `/v3/users/2392d440b8ff4f95b5235d231e9282df` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:15.917549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2392d440b8ff4f95b5235d231e9282df)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:15.919484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:15.969954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2392d440b8ff4f95b5235d231e9282df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:15.976618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2392d440b8ff4f95b5235d231e9282df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:15.977280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2392d440b8ff4f95b5235d231e9282df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:15.977880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2392d440b8ff4f95b5235d231e9282df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:15.985959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2392d440b8ff4f95b5235d231e9282df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.002797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1075/2146] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:15 2022] DELETE /identity/v3/users/2392d440b8ff4f95b5235d231e9282df => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:16.007259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:16.007731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:16.007844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:16.029572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:16.031133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:16.095033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1072/2147] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:16.106607 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:16.109197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:16.182112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e451cbe-cba5-48b5-b20c-def3692043ea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:16.186074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:16.186748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:16.187027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:16.187296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:16.196985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:16.198456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:16.209387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1076/2148] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:16.553044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:16.554780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:16.568271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3eaa17-48cb-46b7-8754-c9494072b314 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:16.573163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:16.573884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:16.574166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:16.574458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] PATH_INFO: `/v3/projects/0da1d4f8e77c40f3b89014e3e440320d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:16.575687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da1d4f8e77c40f3b89014e3e440320d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:16.578254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:16.634393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da1d4f8e77c40f3b89014e3e440320d', 'request_id': 'req-bd25d7fc-c687-4243-9046-3249f8e062f7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.642253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da1d4f8e77c40f3b89014e3e440320d', 'request_id': 'req-bd25d7fc-c687-4243-9046-3249f8e062f7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.648790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da1d4f8e77c40f3b89014e3e440320d', 'request_id': 'req-bd25d7fc-c687-4243-9046-3249f8e062f7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.653488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:16.661492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] The token cache is being invalidate because project 0da1d4f8e77c40f3b89014e3e440320d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:16.662006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.663096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.664588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1073/2149] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:16 2022] DELETE /identity/v3/projects/0da1d4f8e77c40f3b89014e3e440320d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:16.668629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:16.678198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae82fa-0212-4b20-b65b-91e3fa504289 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:16.695167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:16.737140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:16.737788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:16.738072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:16.738348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] PATH_INFO: `/v3/users/2922ccc6340749bdb472e00d663dd95b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:16.739493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2922ccc6340749bdb472e00d663dd95b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:16.741412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:16.789447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2922ccc6340749bdb472e00d663dd95b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.796848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2922ccc6340749bdb472e00d663dd95b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.797566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2922ccc6340749bdb472e00d663dd95b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.798299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2922ccc6340749bdb472e00d663dd95b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.808719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b51b68bf-5872-4426-9a6a-986f5a0e7f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2922ccc6340749bdb472e00d663dd95b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:16.831516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1077/2150] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:16 2022] DELETE /identity/v3/users/2922ccc6340749bdb472e00d663dd95b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:17.114728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.116429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:17.139792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d7fc-c687-4243-9046-3249f8e062f7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.217367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.217998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.218488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.218676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] PATH_INFO: `/v3/projects/94dce9d809fc4947b015a037981779ca` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.220159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94dce9d809fc4947b015a037981779ca)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.222266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.302832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94dce9d809fc4947b015a037981779ca', 'request_id': 'req-efb0d7de-e237-4ff2-b4e9-5b678bb52980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:17.314554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94dce9d809fc4947b015a037981779ca', 'request_id': 'req-efb0d7de-e237-4ff2-b4e9-5b678bb52980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:17.322455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94dce9d809fc4947b015a037981779ca', 'request_id': 'req-efb0d7de-e237-4ff2-b4e9-5b678bb52980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:17.334308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] The token cache is being invalidate because project 94dce9d809fc4947b015a037981779ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:17.334632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:17.335325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-efb0d7de-e237-4ff2-b4e9-5b678bb52980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:17.337036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1074/2151] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:17 2022] DELETE /identity/v3/projects/94dce9d809fc4947b015a037981779ca => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:17.341858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.342324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.342545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.363869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:17.365299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:17.462564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1078/2152] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:17.466540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.469046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.469046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.490614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:17.491832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:17.500977 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1075/2153] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:17.505324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.515840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246e5f-0ccf-40b7-b9fd-7ceacc4d370a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.521324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.521669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.522230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.522474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.523233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:17.523646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.525451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.552200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1079/2154] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:17.556722 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.567210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f490729-9803-45d3-9698-0c9e0d26b7b3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.572505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.573121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.573405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.573674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.574311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.576538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.615935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1076/2155] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:17 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:17.617581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.629020 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6ce0b-da8b-4e3b-89f1-8f779c117200 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.662748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:17.706536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.707139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.707416 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.707695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.708337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.710254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.738350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1080/2156] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:17 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:17.739641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.749946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4371c-458e-4747-b77d-6f3ae81ab85d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.755235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.756016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.756419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.756907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.757724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.759606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.772011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1077/2157] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:17.776011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.786970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40527fb2-31a0-455d-873b-6cb10cefbea5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.791997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.792705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.792988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.793261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.793931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.795623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e04e4-6c35-43e4-807e-8ac163402748 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.810250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1081/2158] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:17.811663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:17.821837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ce99-d48f-417b-a4a3-7fdb5040dd73 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:17.827642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:17.828424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.828711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.828987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] PATH_INFO: `/v3/projects/dbc77b5d0c654c0fab31b8c613476674/users/507bba6838e24d5db8856aa94a811697/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.846946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc77b5d0c654c0fab31b8c613476674, user_id=507bba6838e24d5db8856aa94a811697, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:17.849933 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:17.866981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1078/2159] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:17 2022] PUT /identity/v3/projects/dbc77b5d0c654c0fab31b8c613476674/users/507bba6838e24d5db8856aa94a811697/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:17.877732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:17.878029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:17.878289 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:17.910310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] MFA Rules not processed for user `507bba6838e24d5db8856aa94a811697`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:17.911635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:17.985101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1082/2160] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:17.998821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.000541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.031071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:18.066300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53f89-a8a4-4d60-a79e-c853cd090d68 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.070156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.070801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.071077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.071348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.082294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.083756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6a430-9eb5-4fd2-b3f8-f243d1cc0c68 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.095930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1079/2161] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:18.123222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.186571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a52cf-5eb1-45b0-9c99-16bf3d21c91e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.191343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.192002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.192284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.192564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] PATH_INFO: `/v3/users/507bba6838e24d5db8856aa94a811697` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.193692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507bba6838e24d5db8856aa94a811697)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.195659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.234001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:18.249402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507bba6838e24d5db8856aa94a811697'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.260683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507bba6838e24d5db8856aa94a811697'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.261673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507bba6838e24d5db8856aa94a811697'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.262573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507bba6838e24d5db8856aa94a811697'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.275647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507bba6838e24d5db8856aa94a811697'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.296496 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1083/2162] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:18 2022] DELETE /identity/v3/users/507bba6838e24d5db8856aa94a811697 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:18.302249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.302803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.302896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.322946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:18.324141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.383760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1080/2163] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:18.393934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.395545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.454955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.456767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.465444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33d4da-4a26-4290-b4bf-55a08ab13d46 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.469948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a5e0-f0ce-4da3-b4ac-ddfc16fcc556 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.471332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.472029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.472423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.472659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.476059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.476694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.477197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.477359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] PATH_INFO: `/v3/users/46cbcc750dd0495883f2025a804bb0f5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.478498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cbcc750dd0495883f2025a804bb0f5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.480404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.483253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.484709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4ab89-fae9-406a-b47a-cf679b8d1100 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.496100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1084/2164] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:18.534103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cbcc750dd0495883f2025a804bb0f5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.541748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cbcc750dd0495883f2025a804bb0f5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.542552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cbcc750dd0495883f2025a804bb0f5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.543427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cbcc750dd0495883f2025a804bb0f5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.554124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cbcc750dd0495883f2025a804bb0f5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.586202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1081/2165] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:18 2022] DELETE /identity/v3/users/46cbcc750dd0495883f2025a804bb0f5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:18.587991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.588272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.588610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.611528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:18.612976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.723488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1085/2166] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:18.734836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.736358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.786331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:18.804548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60087363-81f2-4f7d-b05c-f190e6bc704b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.808663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.809368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.809541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.809805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.821157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.822760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779d9a0-6b8e-410a-b9a4-02dbcd1c93e6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.838303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1082/2167] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:18.856125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:18.858184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:18.877851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3058e9c-393d-48f2-a0ab-ab1a68f87f7a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:18.883242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:18.883878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.884159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.884438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] PATH_INFO: `/v3/projects/dbc77b5d0c654c0fab31b8c613476674` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:18.885906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc77b5d0c654c0fab31b8c613476674)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:18.887917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:18.939655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc77b5d0c654c0fab31b8c613476674', 'request_id': 'req-ab18fa6b-f213-4498-aefa-eff3108cb2d4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.941845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:18.948784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc77b5d0c654c0fab31b8c613476674', 'request_id': 'req-ab18fa6b-f213-4498-aefa-eff3108cb2d4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.960145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc77b5d0c654c0fab31b8c613476674', 'request_id': 'req-ab18fa6b-f213-4498-aefa-eff3108cb2d4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.975376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] The token cache is being invalidate because project dbc77b5d0c654c0fab31b8c613476674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:18.975852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.976741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ab18fa6b-f213-4498-aefa-eff3108cb2d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:18.978971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1086/2168] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:18 2022] DELETE /identity/v3/projects/dbc77b5d0c654c0fab31b8c613476674 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:18.983962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:18.984244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:18.984648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.006109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:19.007402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.080680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1083/2169] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:19.089178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.089642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.090121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.111486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:19.112775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.121673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1087/2170] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:19.126838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.137269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7332b1-9528-4f5b-9285-2115d9c3d3e8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.143146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.144004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.144345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.144710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.145533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:19.145891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.147667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.161039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1084/2171] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:19.165553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.176376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696f25b-cb29-4026-baa2-026997c5e1dd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.181207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.181863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.182145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.182436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.183373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.185976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.216100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1088/2172] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:19.220509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.231284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a13d6d-7ad2-4be6-ac6e-669e67945cfc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.295022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.295652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.295940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.296206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.296862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.298819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.327051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1085/2173] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:19.328336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.339213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27df4-fea6-40fe-b8de-c73ad83d2d74 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.344955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.345573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.345854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.346134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.346819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.348463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.361907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1089/2174] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:19.367894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.380660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7de3b7-36dc-48c3-8204-f741803e6960 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.385837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.388067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.388067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.388067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.388067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.390021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.401753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1086/2175] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:19.403106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.405782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.406272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.417481 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920031-4aab-405b-b5fc-b0b69b73231c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.422382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.423239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.423513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.423785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] PATH_INFO: `/v3/projects/5359f4a3a0ad43bcaf5191876513eb1e/users/c631fd48c2774f519a56672fb0246ed8/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.432144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff3dd-7a74-42e3-afd6-2e281c0d4e64 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.437489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.438053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.438330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.438632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] PATH_INFO: `/v3/projects/657ee920907c40d69947f3d2866b92e3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.439740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=657ee920907c40d69947f3d2866b92e3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.440824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5359f4a3a0ad43bcaf5191876513eb1e, user_id=c631fd48c2774f519a56672fb0246ed8, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.441619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.443522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f788c-481d-46bb-90db-40501e79ac92 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.458140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1087/2176] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:19 2022] PUT /identity/v3/projects/5359f4a3a0ad43bcaf5191876513eb1e/users/c631fd48c2774f519a56672fb0246ed8/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:19.469946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.470363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.470671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.494513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657ee920907c40d69947f3d2866b92e3', 'request_id': 'req-03b393cd-2b9b-4a10-adba-0b3f605abe1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:19.499164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] MFA Rules not processed for user `c631fd48c2774f519a56672fb0246ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:19.500375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.501430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657ee920907c40d69947f3d2866b92e3', 'request_id': 'req-03b393cd-2b9b-4a10-adba-0b3f605abe1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:19.509378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657ee920907c40d69947f3d2866b92e3', 'request_id': 'req-03b393cd-2b9b-4a10-adba-0b3f605abe1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:19.510092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:19.522402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] The token cache is being invalidate because project 657ee920907c40d69947f3d2866b92e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:19.522729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:19.523360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03b393cd-2b9b-4a10-adba-0b3f605abe1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:19.524914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1090/2177] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:19 2022] DELETE /identity/v3/projects/657ee920907c40d69947f3d2866b92e3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:19.530229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e32b-2719-4b65-ae0f-e2e5d975dc6a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.530540 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e32b-2719-4b65-ae0f-e2e5d975dc6a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.530814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e32b-2719-4b65-ae0f-e2e5d975dc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.550207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6a75e32b-2719-4b65-ae0f-e2e5d975dc6a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:19.551460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a75e32b-2719-4b65-ae0f-e2e5d975dc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.567946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1088/2178] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:19.580527 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.581918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.613504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1091/2179] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:19.621363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.621653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.621927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.639217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3693fd-31a1-4481-80a8-8ed571314367 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.641225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:19.642433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:19.643013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.643626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.643901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.644167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.651254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1092/2180] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:19.652831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.654940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.655896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.663844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1089/2181] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:19.665138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10e2b7-3d2d-4675-a795-e25466365f93 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.670486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.671093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.671426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.671700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.672369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:19.672673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.674335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.686954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1093/2182] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:19.691292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.701881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4fa94-52fe-4197-a3a4-b48d28b7be14 service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.706559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.707254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.707540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.707802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.708644 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.710627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.742378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1090/2183] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:19.746983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.759560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:19.763679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88db8f-95a4-4c4a-9814-9c9e60d89ad2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.845056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.846002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.846545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.846883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.847734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.850220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.880427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1094/2184] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:19 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:19.884685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.897765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e603-31a3-44ac-bc46-0c81f1e2509f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.902636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.903241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.905152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.905152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.905822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.907442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.921117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1091/2185] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:19.922154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.933365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980960d-9f15-4631-b517-fdb16f11fedc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.938605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.939210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.939494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.939771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.940421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.942147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0bf46-c4b9-49fe-8716-5db5c24316e3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:19.953827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1095/2186] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:19.958579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:19.970830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d3d0-1ad7-45d4-bc3f-86a182bc361d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:19.978085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:19.979297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:19.979297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:19.979525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] PATH_INFO: `/v3/projects/de7fa0618ab54243b01698f5ce78cecb/users/7fba9d49eb4547419595eab1402ea712/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:19.995443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7fa0618ab54243b01698f5ce78cecb, user_id=7fba9d49eb4547419595eab1402ea712, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:19.998224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:20.016239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1092/2187] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:19 2022] PUT /identity/v3/projects/de7fa0618ab54243b01698f5ce78cecb/users/7fba9d49eb4547419595eab1402ea712/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:20.027350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:20.027753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:20.028125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:20.071570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] MFA Rules not processed for user `7fba9d49eb4547419595eab1402ea712`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:20.073106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:20.167408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1096/2188] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:20.195623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:20.198471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:20.287545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95dfee-0580-4124-befc-0b9f5bbe567b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:20.292744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:20.293362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:20.293645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:20.293919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:20.304457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:20.305671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:20.312484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:20.320681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1093/2189] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:23.576990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:23.578994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:23.658135 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64919509-3117-4e54-96b2-ed9b5f623bf2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:23.662645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:23.663269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:23.663544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:23.663815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] PATH_INFO: `/v3/users/76dc93ac31604022a522b47559ab31b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:23.664918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=76dc93ac31604022a522b47559ab31b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:23.666857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:23.680146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:23.714728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76dc93ac31604022a522b47559ab31b5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.723831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76dc93ac31604022a522b47559ab31b5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.725688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76dc93ac31604022a522b47559ab31b5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.725688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76dc93ac31604022a522b47559ab31b5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.733298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-847f032d-3d9f-4376-83d9-3c6e8fa1e39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76dc93ac31604022a522b47559ab31b5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.751191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1097/2190] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:23 2022] DELETE /identity/v3/users/76dc93ac31604022a522b47559ab31b5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:23.923708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:23.937923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f531a1-4416-45ac-8405-8716980ba49a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.005732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.006483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.006870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.007267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] PATH_INFO: `/v3/projects/524d39d780b2426cbacde028e1b061de` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.008426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=524d39d780b2426cbacde028e1b061de)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.010346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.069120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524d39d780b2426cbacde028e1b061de', 'request_id': 'req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:24.079766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524d39d780b2426cbacde028e1b061de', 'request_id': 'req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:24.087318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524d39d780b2426cbacde028e1b061de', 'request_id': 'req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:24.099262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] The token cache is being invalidate because project 524d39d780b2426cbacde028e1b061de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:24.099552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:24.100214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cca16d28-2260-4c4c-abd8-3ffb8d0d49ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:24.101870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1094/2191] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:23 2022] DELETE /identity/v3/projects/524d39d780b2426cbacde028e1b061de => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:24.106721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.107018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.107362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.129207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:24.130556 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.197440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1098/2192] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:24.205752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.206044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.206322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.226663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:24.227865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.235699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1095/2193] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:24.239848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.250717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3ec66-6b89-4b0c-8130-56734ba9b1ed None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.255364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.255967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.256247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.256523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.257168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:24.257475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.259155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.272112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1099/2194] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:24.276406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.285564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c4558-728b-4df0-b843-30ba9f726bdf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.290168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.290791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.291065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.291328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.291943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.293821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.323843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1096/2195] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:24.328697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.338845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff1c1-edbf-4155-9f66-a387ae4d1ccf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.411486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.413862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.413862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.413862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.413862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.416954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.448356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1100/2196] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:24.452468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.464023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedee67-1477-47f9-9f89-6e1cc469af27 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.470276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.471150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.471712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.471894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.472557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.474164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.488380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1097/2197] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:24.492565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.503407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b961247-44d3-4ab7-bd27-4bfde317dfaa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.508246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.508718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.508996 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.509266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.509907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.511552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae6d30-9d43-462c-a9da-390a74f23acb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.513668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:24.523740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1101/2198] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:24.528177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.538095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a960-24d5-476e-9fb7-03b6ce8583b0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.542661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.543304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.543579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.543847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] PATH_INFO: `/v3/projects/c7541b0dd8b14bf495094ddaa132bc2a/users/9ecdfa3c79f84cc29061dd5ebfa60101/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.558816 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7541b0dd8b14bf495094ddaa132bc2a, user_id=9ecdfa3c79f84cc29061dd5ebfa60101, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.562126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.575324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1098/2199] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:24 2022] PUT /identity/v3/projects/c7541b0dd8b14bf495094ddaa132bc2a/users/9ecdfa3c79f84cc29061dd5ebfa60101/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:24.580294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.580766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.580859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.599322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:24.600503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.661808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1102/2200] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:24.672452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.674559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.706171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:24.734999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5bfa-805a-4732-8e40-3f73986e464a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.738697 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.739300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.739580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.739854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.749572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.751054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f02bd-f20d-4714-81f2-29e6cc1517c3 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.760808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1099/2201] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:27.322347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:27.324893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:27.368929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5f12-4c16-4bb0-8609-7b3928c0c968 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:27.389369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:27.389831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:27.390249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:27.390454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] PATH_INFO: `/v3/users/09f37b3a2cb548788794641999068d65` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:27.391544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f37b3a2cb548788794641999068d65)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:27.393616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:27.460745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f37b3a2cb548788794641999068d65'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:27.468605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f37b3a2cb548788794641999068d65'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:27.469794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f37b3a2cb548788794641999068d65'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:27.470766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f37b3a2cb548788794641999068d65'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:27.485402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f37b3a2cb548788794641999068d65'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:27.510433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1103/2202] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:27 2022] DELETE /identity/v3/users/09f37b3a2cb548788794641999068d65 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:27.515718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:27.516574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:27.517378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:27.547375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:27.549435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:27.662291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1100/2203] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:27.674438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:27.675898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:27.769443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6743c-f6ae-4586-8a21-e455a53a9068 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:27.775193 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:27.775849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:27.776330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:27.781171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:27.797914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:27.799482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:27.820404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1104/2204] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:28.287700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:28.289367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:28.333652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66b441-acbe-4653-bcd7-321e30e57123 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:28.351288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:28.351936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:28.352215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:28.352553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] PATH_INFO: `/v3/projects/d6b541ed034347bbb5393caafac101e7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:28.354222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b541ed034347bbb5393caafac101e7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:28.356191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:28.522223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b541ed034347bbb5393caafac101e7', 'request_id': 'req-f6c57605-0491-4acd-9fa8-7a3f54e77110'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.542475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b541ed034347bbb5393caafac101e7', 'request_id': 'req-f6c57605-0491-4acd-9fa8-7a3f54e77110'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.567413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b541ed034347bbb5393caafac101e7', 'request_id': 'req-f6c57605-0491-4acd-9fa8-7a3f54e77110'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.589476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] The token cache is being invalidate because project d6b541ed034347bbb5393caafac101e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:28.590033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.590891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.607099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1101/2205] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:28 2022] DELETE /identity/v3/projects/d6b541ed034347bbb5393caafac101e7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:28.608407 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:28.619110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08cd6-7fda-4eca-95cd-ee9d029cdf72 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:28.632322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:28.740077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:28.741060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:28.741346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:28.744398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] PATH_INFO: `/v3/users/4b32dc8f833d414f8167aa2bbc551102` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:28.746919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b32dc8f833d414f8167aa2bbc551102)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:28.749412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:28.817412 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b32dc8f833d414f8167aa2bbc551102'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.829597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b32dc8f833d414f8167aa2bbc551102'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.831223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b32dc8f833d414f8167aa2bbc551102'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.832663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b32dc8f833d414f8167aa2bbc551102'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.848433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9104e7e9-78b4-4129-89f7-1ce928816fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b32dc8f833d414f8167aa2bbc551102'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:28.875640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1105/2206] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:28 2022] DELETE /identity/v3/users/4b32dc8f833d414f8167aa2bbc551102 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:28.881776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:28.888743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:28.920624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57605-0491-4acd-9fa8-7a3f54e77110 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.023838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:29.048086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.049174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.049655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.050270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] PATH_INFO: `/v3/users/679d4c1ba4d44e2f9435f9a318f302cd` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.051835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=679d4c1ba4d44e2f9435f9a318f302cd)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.054594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:29.177256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679d4c1ba4d44e2f9435f9a318f302cd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.205538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679d4c1ba4d44e2f9435f9a318f302cd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.207778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679d4c1ba4d44e2f9435f9a318f302cd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.208872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679d4c1ba4d44e2f9435f9a318f302cd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.222138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679d4c1ba4d44e2f9435f9a318f302cd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.280081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1102/2207] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:28 2022] DELETE /identity/v3/users/679d4c1ba4d44e2f9435f9a318f302cd => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:29.284806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.285197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.285549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.311201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:29.314366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.458523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1106/2208] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:29.475044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.476776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.507784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.512754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.522397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eb3b6-ed08-40b9-9b3f-7b4b02a91064 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.564388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.565441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.566050 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.566323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] PATH_INFO: `/v3/projects/eb755534312141f7b62968d71a337a5f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.567524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb755534312141f7b62968d71a337a5f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.569505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:29.602328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:29.626505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb755534312141f7b62968d71a337a5f', 'request_id': 'req-a9329fc9-ef9e-4114-be4a-2250734f7e7d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.632960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213052a-a580-4e32-8da9-8e5f6953b5ba None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.636574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb755534312141f7b62968d71a337a5f', 'request_id': 'req-a9329fc9-ef9e-4114-be4a-2250734f7e7d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.644699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb755534312141f7b62968d71a337a5f', 'request_id': 'req-a9329fc9-ef9e-4114-be4a-2250734f7e7d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.657922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] The token cache is being invalidate because project eb755534312141f7b62968d71a337a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:29.658210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.658888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a9329fc9-ef9e-4114-be4a-2250734f7e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:29.660619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1103/2209] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:29 2022] DELETE /identity/v3/projects/eb755534312141f7b62968d71a337a5f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:29.661960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3692-8ec2-44ba-adfb-8d690e2b43f0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.662388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3692-8ec2-44ba-adfb-8d690e2b43f0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.663063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3692-8ec2-44ba-adfb-8d690e2b43f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.699728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3c2e3692-8ec2-44ba-adfb-8d690e2b43f0 None None] MFA Rules not processed for user `9ecdfa3c79f84cc29061dd5ebfa60101`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:29.699987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.701091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.701374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.701766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.701935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c2e3692-8ec2-44ba-adfb-8d690e2b43f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.705111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.758374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:29.782818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.784256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:29.786168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1104/2210] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:29.787674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb581f6b-1690-4f12-9bdc-d048386ff0ad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.788059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb581f6b-1690-4f12-9bdc-d048386ff0ad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.788436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb581f6b-1690-4f12-9bdc-d048386ff0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.794687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1107/2211] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:29.803759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.805579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.814143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bb581f6b-1690-4f12-9bdc-d048386ff0ad None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:29.816092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb581f6b-1690-4f12-9bdc-d048386ff0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.829144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1105/2212] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:29.838593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.839258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.839999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.867675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:29.869004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:29.881948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1106/2213] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:29.883701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.895223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d2feb-a01c-4655-8302-275d8c60816d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.900230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.900892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.901175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.901444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.902219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:29.902586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.905176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:29.921812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1107/2214] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:44:29.923965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.935495 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262150e-8584-4f45-b0b6-d23af40ac4ae admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.942474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.942474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.942474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.942474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.943157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.947057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:29.954103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaae6d-3984-4dde-a2dc-d0ea50b0cf91 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:29.963469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:29.964690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:29.965427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:29.967241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:29.987291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1108/2215] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:29.992840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:29.995092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:29.997225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.039492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1108/2216] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:30.098814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef705-f7d7-452d-9d10-60a4e57c666b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.103776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.104395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.104673 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.104950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.105811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.108931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.163536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1109/2217] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:29 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:30.164927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.177991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a47e-7d46-45d2-8e96-a50a76e86b2d service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.190868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.193627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.194098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.195350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.201183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.205757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.223107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1109/2218] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:30.235881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.239447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.245613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.268431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141518e5-fc3e-4499-a1ea-7586e412cbb9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.269562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c37fd-4900-4450-9237-27e7036d3eb7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.275184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.276022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.276450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.276732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] PATH_INFO: `/v3/projects/50604b83540046219a9ec969664f4767` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.276844 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.277587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.278001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.278889 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.279510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=50604b83540046219a9ec969664f4767)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.279730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.282528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.282736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.296339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1110/2219] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:30.299297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:30.300706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.312280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ac35e-88f6-47c5-bc21-c9e0f1878eb5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.319019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.319709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.319991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.320265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] PATH_INFO: `/v3/projects/30e41e11ec594c50b5570c68e505520c/users/3145d3d8bfdb45d78dad4d8ad4e3913d/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.343204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e41e11ec594c50b5570c68e505520c, user_id=3145d3d8bfdb45d78dad4d8ad4e3913d, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.346383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da912604-d8d0-486e-8022-54fdc8799335 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.378002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1111/2220] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:30 2022] PUT /identity/v3/projects/30e41e11ec594c50b5570c68e505520c/users/3145d3d8bfdb45d78dad4d8ad4e3913d/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:30.378236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50604b83540046219a9ec969664f4767', 'request_id': 'req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.392875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50604b83540046219a9ec969664f4767', 'request_id': 'req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.394801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.395470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.395980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.405927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50604b83540046219a9ec969664f4767', 'request_id': 'req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.433907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] The token cache is being invalidate because project 50604b83540046219a9ec969664f4767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:30.434205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.435559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.437985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1110/2221] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:30 2022] DELETE /identity/v3/projects/50604b83540046219a9ec969664f4767 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:30.442052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.443726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] MFA Rules not processed for user `3145d3d8bfdb45d78dad4d8ad4e3913d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:30.444475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.444965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.508471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:30.545285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1112/2222] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:30.551691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.554448 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.556172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148a67-c1b8-4cf5-baa2-f5e8033ddfde admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.654778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.655708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.656071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.656440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] PATH_INFO: `/v3/users/eff542f7e5dc4be9801d6994a268420f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.662949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff542f7e5dc4be9801d6994a268420f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.665360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.756087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fd9ac-0428-4ad6-b3d9-a40049bfa126 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.793698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:30.797003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff542f7e5dc4be9801d6994a268420f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.799156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.801588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.802121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.802511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.803089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.806225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.830551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff542f7e5dc4be9801d6994a268420f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.831928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff542f7e5dc4be9801d6994a268420f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.841427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff542f7e5dc4be9801d6994a268420f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.853107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff542f7e5dc4be9801d6994a268420f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.867604 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1113/2223] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/projects => generated 385 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:30.886692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.890174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.895341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1111/2224] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:30 2022] DELETE /identity/v3/users/eff542f7e5dc4be9801d6994a268420f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:31.026952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dc63-b664-4c73-bb15-8623ef98ac38 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.106155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.107190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.107498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.107772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.108577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.110747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.156717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1114/2225] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:31.157877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.172118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab035-b902-4f22-9a11-3dd46c8e1a8c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.177456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.178260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.178679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.179022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.179809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.181917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.195963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1112/2226] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:31.200459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.206576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:31.213684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895034-fe42-484c-b234-9902386cd585 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.220751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.221763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.222355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.222748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.223712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.225459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.238012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1115/2227] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:31.242245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.258063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc04c3-a1d0-44d9-abd2-99a9b7e4380a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.271884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.273377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.273686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.273960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] PATH_INFO: `/v3/projects/e4d56ec9666c41179fa93929abb8bc47/users/4856609e4dd34f968e33876cd5509f18/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.300335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d56ec9666c41179fa93929abb8bc47, user_id=4856609e4dd34f968e33876cd5509f18, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.305434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a17a01-c173-410d-8cd8-c5b0c1e2c9c9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.332358 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.334526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.343534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1113/2228] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:31 2022] PUT /identity/v3/projects/e4d56ec9666c41179fa93929abb8bc47/users/4856609e4dd34f968e33876cd5509f18/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:31.354842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.355140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.355422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.360906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3095-c927-407c-8ca0-e7e7108661f5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.402787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] MFA Rules not processed for user `4856609e4dd34f968e33876cd5509f18`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:31.404762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.442457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.442900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.443186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.443459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] PATH_INFO: `/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.444653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8b92ac2ba44579b4ebb3ee889b032d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.446619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.512468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1114/2229] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:31.517202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.526904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8b92ac2ba44579b4ebb3ee889b032d', 'request_id': 'req-aaf44539-8471-4a3b-8e2b-87aa31636bd7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:31.532347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef1e46-c121-4727-b590-522661b45c01 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.540007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8b92ac2ba44579b4ebb3ee889b032d', 'request_id': 'req-aaf44539-8471-4a3b-8e2b-87aa31636bd7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:31.556710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8b92ac2ba44579b4ebb3ee889b032d', 'request_id': 'req-aaf44539-8471-4a3b-8e2b-87aa31636bd7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:31.574243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] The token cache is being invalidate because project ca8b92ac2ba44579b4ebb3ee889b032d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:31.574748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:31.576165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aaf44539-8471-4a3b-8e2b-87aa31636bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:31.578082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1116/2230] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:31 2022] DELETE /identity/v3/projects/ca8b92ac2ba44579b4ebb3ee889b032d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:31.583210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df802d-890d-4f92-b699-e33f56a77947 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.583510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df802d-890d-4f92-b699-e33f56a77947 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.584243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df802d-890d-4f92-b699-e33f56a77947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.611058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.612124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-16df802d-890d-4f92-b699-e33f56a77947 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:31.612302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.612302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.612302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.612876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.613044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-16df802d-890d-4f92-b699-e33f56a77947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.614840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.637043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1117/2231] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:31.646442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.646814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.647161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.671511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1115/2232] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:31.672452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.674123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.677065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:31.679279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.679687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:31.774064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc278d-b2d6-4700-a34a-ab9e5e24f22b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.775156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1118/2233] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:31.779271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.785694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.785892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.786206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.786664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.787385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.789594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.790509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db34c-89c6-47fb-9df0-e69ab52f7e1b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.796425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.797114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.797394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.797667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.798343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:31.798787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.800508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.816068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1116/2234] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:31.819984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.832155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d24-7314-4ef8-84d3-dcf00e74745e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.833878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1119/2235] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:44:31.838377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.841495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.842216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.842596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.842940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.843686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.844296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:31.846353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.850521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df58ca7-77f5-49bb-9a74-f8862e07634f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.858998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.860068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.860714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.861426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.862135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.862897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1117/2236] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:31.863645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.864549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.874296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f55e1-1674-4e8b-be6f-93b2424bdfef admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.879254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.880075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.880319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.880596 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] PATH_INFO: `/v3/projects/f93b7f12345c475ba90fe805ef9ca539/users/ac08a2e5d6694f2195cd28d29495cf11/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.899682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93b7f12345c475ba90fe805ef9ca539, user_id=ac08a2e5d6694f2195cd28d29495cf11, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.902470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.926013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1120/2237] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:31.931119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1118/2238] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:31 2022] PUT /identity/v3/projects/f93b7f12345c475ba90fe805ef9ca539/users/ac08a2e5d6694f2195cd28d29495cf11/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:31.931301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.935883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.030446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58f16e-f73b-4d91-a3b6-9927afd0cf52 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.035756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.036367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.036920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.037204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.037960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.039662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.054675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1119/2239] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:32.058903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.070962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c38c4f-ef15-44b4-baa0-a82cfbff0121 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.072246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684a39f-e33e-4ae4-a44d-4accab88f901 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.076745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.077452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.077731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.078002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] PATH_INFO: `/v3/projects/f93b7f12345c475ba90fe805ef9ca539/users/ac08a2e5d6694f2195cd28d29495cf11/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.080105 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93b7f12345c475ba90fe805ef9ca539, user_id=ac08a2e5d6694f2195cd28d29495cf11, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.083099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.083521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.084580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.084984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.085474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.087999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.093515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af869f-0202-44c9-912d-53b975d9a633 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.096494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1120/2240] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:32 2022] PUT /identity/v3/projects/f93b7f12345c475ba90fe805ef9ca539/users/ac08a2e5d6694f2195cd28d29495cf11/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:32.145128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1121/2241] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:32.151848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.186467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.186467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.186467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.238661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] MFA Rules not processed for user `ac08a2e5d6694f2195cd28d29495cf11`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:32.239939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:32.275303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:32.299448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901903ad-79a5-4eae-9ebc-c8cf0aec4222 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.305059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.305709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.305945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.306212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.307037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.308811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.322820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1121/2242] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:32.323894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.334732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393c2-755f-4db5-a602-489f3e91d598 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.342142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.342971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.343136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.343776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.344685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.346861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f09ea-1c38-4712-bf70-d9401fe2ec41 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.365566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1122/2243] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:32.367367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb60a7-b837-4a4f-bc7a-836238e37aee None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.367758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb60a7-b837-4a4f-bc7a-836238e37aee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.368441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb60a7-b837-4a4f-bc7a-836238e37aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.379730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1122/2244] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:32.381562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.393954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b4d64-025c-4902-85cf-351fc8017b0a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.394078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-77bb60a7-b837-4a4f-bc7a-836238e37aee None None] MFA Rules not processed for user `9ecdfa3c79f84cc29061dd5ebfa60101`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:32.395695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77bb60a7-b837-4a4f-bc7a-836238e37aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:32.402381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.403863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.404133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.404411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] PATH_INFO: `/v3/projects/bfeede09780f4d7f8136c5f06001412f/users/8e28c435fcc847c89be5397ed0604212/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.429297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfeede09780f4d7f8136c5f06001412f, user_id=8e28c435fcc847c89be5397ed0604212, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.429297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.453321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1123/2245] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:32 2022] PUT /identity/v3/projects/bfeede09780f4d7f8136c5f06001412f/users/8e28c435fcc847c89be5397ed0604212/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:32.453321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.455258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:32.479001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1123/2246] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:32.480283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.480564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.480866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.502240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:32.536353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] MFA Rules not processed for user `8e28c435fcc847c89be5397ed0604212`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:32.542456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:32.553214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8156e8-e8ca-40a9-b5d9-225cd6a3c7eb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.557717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.559593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.560027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.560474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.562782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:32.643084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1124/2247] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:32.647605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.668229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.684832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.709668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1124/2248] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:32.751598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db9a64-0cc9-4b70-974f-e00b09487426 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.764148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.764898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.765081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.765320 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.765955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.767976 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:32.774664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:32.809669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1125/2249] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:32 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:32.811236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:32.828384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39859-2947-4f38-ada7-30cd9cc80062 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:32.953089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:32.954216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:32.954216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:32.954389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:32.954962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:32.958623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.012880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1125/2250] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:32 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:33.016353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.030206 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c80ea-4e69-4cac-b1c6-293dbcb8099d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.047287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.048386 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.048672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.049327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.049707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.051619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.066532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1126/2251] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:33.070724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.095546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15618087-111a-478a-88ce-d0b429c2a7c8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.106195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.107394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.107679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.107957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] PATH_INFO: `/v3/projects/be437018df094c219a6b39210399e1f7/users/c57caa65e3bf48c9b36d156f5fd5a69a/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.161438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be437018df094c219a6b39210399e1f7, user_id=c57caa65e3bf48c9b36d156f5fd5a69a, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.164791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.183973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1126/2252] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:33 2022] PUT /identity/v3/projects/be437018df094c219a6b39210399e1f7/users/c57caa65e3bf48c9b36d156f5fd5a69a/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:33.188557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.201226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490091c-e2e2-42f4-8d3f-0f4f7ca37014 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.304930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.305664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.306035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.306533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.307255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.310125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20596454-1c0a-4172-bd5f-bb171f7f7e4b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.330794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1127/2253] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:33.331507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.346320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a955d49-0f48-4d8e-b825-63c59aeca759 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.354828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.355819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.356282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.356700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] PATH_INFO: `/v3/projects/be437018df094c219a6b39210399e1f7/users/c57caa65e3bf48c9b36d156f5fd5a69a/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.359295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be437018df094c219a6b39210399e1f7, user_id=c57caa65e3bf48c9b36d156f5fd5a69a, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.362749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.407772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1127/2254] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:33 2022] PUT /identity/v3/projects/be437018df094c219a6b39210399e1f7/users/c57caa65e3bf48c9b36d156f5fd5a69a/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:33.418292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.419823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.420113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.456769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] MFA Rules not processed for user `c57caa65e3bf48c9b36d156f5fd5a69a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:33.458324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:33.635713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1128/2255] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:33.649142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.695331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:33.715878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd598-9a15-4b81-8f42-090814f8c9b4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.729062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.729988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.730624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.731078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.743413 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.745114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.767694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1128/2256] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:33.854310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:33.867014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6b81-8f27-481f-a79b-606e07cbd179 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:33.870803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:33.871501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:33.872357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:33.872668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:33.951337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:33.955156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:33.956738 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:33.969393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1129/2257] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:35.545079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:35.554996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662717e-4175-472e-a7c8-2ac3629853bb service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:35.560389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:35.561206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:35.561552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:35.562035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:35.563973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:35.627140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:35.628567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:35.638175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1129/2258] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:36.243194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:36.245031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:36.257309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a190cc-6ec3-4774-9cb0-15e3e6084cf4 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:36.268904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:36.269644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:36.270487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:36.270487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:36.371558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:36.373216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a10fa-ac05-4af0-a4a5-3f1efbb7c495 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:36.393963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1130/2259] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:37.183470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:37.185027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:37.253159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea681-0b9c-4e2c-9a16-928fdd4af2f4 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:37.257860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:37.259759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:37.259759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:37.259759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] PATH_INFO: `/v3/users/7fba9d49eb4547419595eab1402ea712` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:37.260507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fba9d49eb4547419595eab1402ea712)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:37.262981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:37.318027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fba9d49eb4547419595eab1402ea712'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.325918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fba9d49eb4547419595eab1402ea712'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.326731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fba9d49eb4547419595eab1402ea712'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.327461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fba9d49eb4547419595eab1402ea712'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.336002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fba9d49eb4547419595eab1402ea712'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.350305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:37.357351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1130/2260] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:37 2022] DELETE /identity/v3/users/7fba9d49eb4547419595eab1402ea712 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:37.362252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:37.362753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:37.363008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:37.387651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:37.388929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:37.456009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1131/2261] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:37.466202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:37.467680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:37.531277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685516d-2f70-4a42-b5b9-bfa8fe3c05d9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:37.535217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:37.535831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:37.536114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:37.536388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:37.547542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:37.549458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18400498-7ed7-4b03-afeb-c74968003f7b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:37.559691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1131/2262] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:37.849299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:37.851423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:37.868468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c3a01-861e-483b-bf11-53c347c0a88a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:37.874947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:37.875934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:37.876357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:37.876773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] PATH_INFO: `/v3/projects/de7fa0618ab54243b01698f5ce78cecb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:37.878303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7fa0618ab54243b01698f5ce78cecb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:37.880942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:37.935432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7fa0618ab54243b01698f5ce78cecb', 'request_id': 'req-3565ce3a-7c59-49a5-a8de-ded994c5629e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.943546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7fa0618ab54243b01698f5ce78cecb', 'request_id': 'req-3565ce3a-7c59-49a5-a8de-ded994c5629e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.956810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7fa0618ab54243b01698f5ce78cecb', 'request_id': 'req-3565ce3a-7c59-49a5-a8de-ded994c5629e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.973574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] The token cache is being invalidate because project de7fa0618ab54243b01698f5ce78cecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:37.974141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.974829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3565ce3a-7c59-49a5-a8de-ded994c5629e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:37.977300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1132/2263] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:37 2022] DELETE /identity/v3/projects/de7fa0618ab54243b01698f5ce78cecb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:37.982423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:37.982851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:37.983216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.003098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:38.004419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:38.075987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1132/2264] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:38.085170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.085637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.086058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.107466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:38.108758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:38.117751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1133/2265] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:38.123651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.133313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a127-cd4d-4ca3-9266-7788e181bc37 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.138362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.139028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.139303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.139572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.140251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:38.140523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.142135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.154642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1133/2266] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:38.158966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.164657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:38.168833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a18813-1767-45cb-ae5a-549ba9f4dbca None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.175121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.175906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.176354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.176680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.177335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.179485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.209594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1134/2267] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:38 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:38.212612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.223308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f47c5-af24-4dc9-8c62-0bea2cebe871 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.284775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.285310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.285588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.285859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.286519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.288473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.314159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1134/2268] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:38 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:38.318704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.329544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e8bb4-8042-4c2a-81d2-19d147558e7b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.334434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.335025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.335307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.335599 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.336228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.337806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.347906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1135/2269] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:38.352086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.364445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aebe5-8c3a-466e-9592-c97b988bcff0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.369365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.369973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.370252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.370554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.371203 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.373328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88967d3c-1952-4f96-bb2a-1229b94b2be6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.385146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1135/2270] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:38.390032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.400467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b995a5c-c893-40b8-892a-e1f9c5164e64 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.406395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.407173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.407460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.407729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] PATH_INFO: `/v3/projects/b62283fd6a124de0a32ad8f586cf997e/users/0ca903cc98a24621b7c3be488e3e7c5e/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.423117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62283fd6a124de0a32ad8f586cf997e, user_id=0ca903cc98a24621b7c3be488e3e7c5e, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.425759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.440770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1136/2271] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:38 2022] PUT /identity/v3/projects/b62283fd6a124de0a32ad8f586cf997e/users/0ca903cc98a24621b7c3be488e3e7c5e/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:38.446310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.446717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.447062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.464122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:38.471099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:38.473083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:38.533466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1136/2272] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:38.543248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:38.545003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:38.601430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0e61d-bc63-4437-afab-d5c803f6a65f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:38.605241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:38.605844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:38.606125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:38.606399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:38.615381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:38.616802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890d05-ee4c-4134-b69e-4918f3c9e2ad service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:38.627223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1137/2273] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:43.467293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:43.469042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:43.505801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d7d2-4a6f-4966-bc3a-a5a01c9104e2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:43.510629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:43.511719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:43.512230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:43.512677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] PATH_INFO: `/v3/users/492bbe43ecc343cf94b5b77ed5885b2c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:43.520569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] RBAC: Authorizing `identity:delete_user(user_id=492bbe43ecc343cf94b5b77ed5885b2c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:43.522987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:43.625235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '492bbe43ecc343cf94b5b77ed5885b2c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:43.639178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '492bbe43ecc343cf94b5b77ed5885b2c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:43.642753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '492bbe43ecc343cf94b5b77ed5885b2c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:43.644679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '492bbe43ecc343cf94b5b77ed5885b2c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:43.670563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '492bbe43ecc343cf94b5b77ed5885b2c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:43.718081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1137/2274] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:43 2022] DELETE /identity/v3/users/492bbe43ecc343cf94b5b77ed5885b2c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:43.719482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:43.719960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:43.720683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:43.745926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:43.747353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:43.841535 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1138/2275] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:43.851475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:43.853095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:43.928798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b1f5-f3e7-42f9-a96f-380a4f69201e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:43.938552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:43.939445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:43.939878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:43.940914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:43.957590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:43.960005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef3b89-77c0-4285-bc4a-39493ce0abd4 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:44.009158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1138/2276] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:44.535216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:44.544396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:44.562369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b738d0-c6c1-4683-8454-3132631b07fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:44.589581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:44.590404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:44.590814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:44.591186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] PATH_INFO: `/v3/projects/de1a6051dcf943b79624ac54a9cf6855` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:44.592800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1a6051dcf943b79624ac54a9cf6855)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:44.595062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:44.672168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:44.677160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1a6051dcf943b79624ac54a9cf6855', 'request_id': 'req-14550293-2d0f-46c9-afd9-a357fadf3eab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:44.690506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1a6051dcf943b79624ac54a9cf6855', 'request_id': 'req-14550293-2d0f-46c9-afd9-a357fadf3eab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:44.701831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1a6051dcf943b79624ac54a9cf6855', 'request_id': 'req-14550293-2d0f-46c9-afd9-a357fadf3eab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:44.716581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] The token cache is being invalidate because project de1a6051dcf943b79624ac54a9cf6855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:44.717555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:44.718245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14550293-2d0f-46c9-afd9-a357fadf3eab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:44.720510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1139/2277] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:44 2022] DELETE /identity/v3/projects/de1a6051dcf943b79624ac54a9cf6855 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:44.726186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:44.727350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:44.727888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:44.755931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:44.761567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:44.796077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:44.874474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1139/2278] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:44.874615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:44.874845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:44.875126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:44.916356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:44.917994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:44.935275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1140/2279] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:44.939144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:44.951069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a9e3c-a108-4d3f-8a9c-643aa54d1fc8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:44.956874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:44.957661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:44.958480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:44.959074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:44.959962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:44.960442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:44.962565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:44.977013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:44.978604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.000766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabed84-59ce-4b39-990e-d93fb703a612 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.002300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1140/2280] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:45.003878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.016077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.017131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.017564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.017991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] PATH_INFO: `/v3/users/3145d3d8bfdb45d78dad4d8ad4e3913d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.019383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3145d3d8bfdb45d78dad4d8ad4e3913d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.028174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.032172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d938bd-7f1e-41c8-8a7d-22005b48177d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.042657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.046578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.047205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.047635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.048460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.051356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.111924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1141/2281] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:45 2022] POST /identity/v3/projects => generated 361 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:45.124613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.139611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3145d3d8bfdb45d78dad4d8ad4e3913d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:45.159104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3145d3d8bfdb45d78dad4d8ad4e3913d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:45.160201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3145d3d8bfdb45d78dad4d8ad4e3913d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:45.161123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3145d3d8bfdb45d78dad4d8ad4e3913d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:45.179091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1d22810-12ec-44e9-93f5-8787a9d15e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3145d3d8bfdb45d78dad4d8ad4e3913d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:45.241276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1141/2282] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:44 2022] DELETE /identity/v3/users/3145d3d8bfdb45d78dad4d8ad4e3913d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:45.247140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.247437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.247756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.281571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:45.283309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.310700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab607b92-4b33-417f-be7e-04237d125bed admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.378762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1142/2283] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:45.391676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.395210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.485237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.485975 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.490830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.491233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.492008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.493992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.500919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:45.529318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48460-5f4d-4688-acd0-b8aaeba42af0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.537046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.541031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.541031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.541031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.542027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1142/2284] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:45 2022] POST /identity/v3/users => generated 281 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:45.543402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.549322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.583921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe285df-31bd-493d-85be-d13ece41efb0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.586401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.589989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.591245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.591529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.591803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.592459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.592701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.594091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.613425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1143/2285] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:45.614157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1143/2286] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:45.615133 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.630840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ac142-ab39-418c-902f-cd36a625d522 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.646751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.649771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.650394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.650897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.651886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.655464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821258a4-843c-434e-846c-c32327e8e418 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.671343 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1144/2287] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:45.676915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.693197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87614c-aac2-41c5-af69-765b93d5ffc6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:45.701422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:45.702256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.708491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.708927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] PATH_INFO: `/v3/projects/fd7c3947ad0245658d115c7c8b13997e/users/a16a0b5e09e9403f8ec2ede931ffc5ed/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.737339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:45.748547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7c3947ad0245658d115c7c8b13997e, user_id=a16a0b5e09e9403f8ec2ede931ffc5ed, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:45.753985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:45.774134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1144/2288] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:45 2022] PUT /identity/v3/projects/fd7c3947ad0245658d115c7c8b13997e/users/a16a0b5e09e9403f8ec2ede931ffc5ed/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:45.785961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:45.786516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:45.786516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:45.825307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] MFA Rules not processed for user `a16a0b5e09e9403f8ec2ede931ffc5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:45.827251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.860075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:45.861943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:45.961801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1145/2289] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:45.997258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa9f36-fc66-488c-b504-6674e728eb9c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.001063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.002185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.011056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.015257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.016207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.016645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] PATH_INFO: `/v3/users/8e28c435fcc847c89be5397ed0604212` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.018308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e28c435fcc847c89be5397ed0604212)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.020638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.101394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e28c435fcc847c89be5397ed0604212'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.114614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4f915-10f9-4b76-804a-f395fdd528a6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.118834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e28c435fcc847c89be5397ed0604212'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.119113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.119638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e28c435fcc847c89be5397ed0604212'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.119751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.119999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.120272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e28c435fcc847c89be5397ed0604212'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.120374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.122647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.130825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e28c435fcc847c89be5397ed0604212'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.135196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.136839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe30e4-2229-4c78-bf90-2171520f22d6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.149179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1146/2290] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:46.152795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0ec9-91ef-4a04-a079-6fdcdd2e5373 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.153075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0ec9-91ef-4a04-a079-6fdcdd2e5373 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.153347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0ec9-91ef-4a04-a079-6fdcdd2e5373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.157968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1145/2291] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:45 2022] DELETE /identity/v3/users/8e28c435fcc847c89be5397ed0604212 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:46.158961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.160638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.183729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b05b0ec9-91ef-4a04-a079-6fdcdd2e5373 None None] MFA Rules not processed for user `0ca903cc98a24621b7c3be488e3e7c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:46.185507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b05b0ec9-91ef-4a04-a079-6fdcdd2e5373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.243153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b63b7-b997-458a-9b87-a4683405e521 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.249569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.250571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.253100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.253505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] PATH_INFO: `/v3/projects/30e41e11ec594c50b5570c68e505520c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.254866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e41e11ec594c50b5570c68e505520c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.256938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.269562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:46.275228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:46.292636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1147/2292] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:46.294505 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.294793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.295064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.316771 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:46.318038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.335242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e41e11ec594c50b5570c68e505520c', 'request_id': 'req-9eb7e83f-8315-4569-ae19-663e2131f439'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.348115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e41e11ec594c50b5570c68e505520c', 'request_id': 'req-9eb7e83f-8315-4569-ae19-663e2131f439'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.359319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e41e11ec594c50b5570c68e505520c', 'request_id': 'req-9eb7e83f-8315-4569-ae19-663e2131f439'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.374091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] The token cache is being invalidate because project 30e41e11ec594c50b5570c68e505520c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:46.374385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.375491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.377317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1146/2293] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:46 2022] DELETE /identity/v3/projects/30e41e11ec594c50b5570c68e505520c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:46.379291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.382329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.402506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1148/2294] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:46.403458 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.404919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.418178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013d351-4e43-4367-b401-bd3991add6e2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.422994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.423605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.423886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.424161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.424800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.426740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.468792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7e83f-8315-4569-ae19-663e2131f439 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.469247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1149/2295] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:46 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:46.470380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.472645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.548056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.549025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.549388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.549984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] PATH_INFO: `/v3/users/a16a0b5e09e9403f8ec2ede931ffc5ed` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.551562 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16a0b5e09e9403f8ec2ede931ffc5ed)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.553805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.573081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189bb30-349c-430a-bd4a-3c98ea731a68 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.579690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.580868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.581156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.581843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] PATH_INFO: `/v3/users/4856609e4dd34f968e33876cd5509f18` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.584325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4856609e4dd34f968e33876cd5509f18)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.586454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.628843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16a0b5e09e9403f8ec2ede931ffc5ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.630382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:46.637093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16a0b5e09e9403f8ec2ede931ffc5ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.638647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16a0b5e09e9403f8ec2ede931ffc5ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.639422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16a0b5e09e9403f8ec2ede931ffc5ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.647921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16a0b5e09e9403f8ec2ede931ffc5ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.666977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4856609e4dd34f968e33876cd5509f18'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.670750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1147/2296] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:46 2022] DELETE /identity/v3/users/a16a0b5e09e9403f8ec2ede931ffc5ed => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:46.671506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.673058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.674721 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4856609e4dd34f968e33876cd5509f18'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.675422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4856609e4dd34f968e33876cd5509f18'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.676802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4856609e4dd34f968e33876cd5509f18'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.685894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4856609e4dd34f968e33876cd5509f18'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:46.695118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:46.711019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1150/2297] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:46 2022] DELETE /identity/v3/users/4856609e4dd34f968e33876cd5509f18 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:46.711019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.711019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.772776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e7bd2-4081-483d-a066-4f9eaf4850ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.809213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e792c4-615e-4150-807e-5fb5bbffc3cb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.814963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.815892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.816167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.816575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.817928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.821315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164bd6f-f298-433c-a4a0-ef2d798bec7d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.853037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.853671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.853948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.854254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.858716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1151/2298] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:46 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:46.860225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba326-a096-4922-beef-70b305dc4996 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.860511 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba326-a096-4922-beef-70b305dc4996 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.860785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba326-a096-4922-beef-70b305dc4996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.863729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.882434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-37dba326-a096-4922-beef-70b305dc4996 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:46.883677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37dba326-a096-4922-beef-70b305dc4996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.885087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.886780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.893047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1152/2299] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:46.894274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba326-a096-4922-beef-70b305dc4996 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.910119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba326-a096-4922-beef-70b305dc4996 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.916018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1148/2300] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:46.917494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.917746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.919021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.919149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.919514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.919872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.920736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.922861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.935646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1153/2301] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:46.940494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.956417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f408e-17b5-43d1-837e-124f0b9b0f89 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.960111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f486e-c2e9-4a47-aeb4-f5dd1d2bf2db service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.963562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.964953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.966057 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.966635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] PATH_INFO: `/v3/projects/113b0f00d7b04db0b1f3ae9cba916275/users/3db3c6d0b14a497d9aeee6ea7ab09546/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.967627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.968647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.969048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.969316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.989284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.990858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.991310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113b0f00d7b04db0b1f3ae9cba916275, user_id=3db3c6d0b14a497d9aeee6ea7ab09546, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.994948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.009806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1149/2302] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:47.017147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1154/2303] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:46 2022] PUT /identity/v3/projects/113b0f00d7b04db0b1f3ae9cba916275/users/3db3c6d0b14a497d9aeee6ea7ab09546/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:47.022639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.048004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791267f-eec3-4f4a-aa2c-229f083b8301 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:47.151093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:47.152583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:47.153521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:47.153814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:47.154716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:47.156821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.188085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1150/2304] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:47.191724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.217514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0612cb-4424-4ed3-8f40-414d8c3efec1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:47.226885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:47.227681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:47.227956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:47.228241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] PATH_INFO: `/v3/projects/113b0f00d7b04db0b1f3ae9cba916275/users/3db3c6d0b14a497d9aeee6ea7ab09546/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:47.233658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=113b0f00d7b04db0b1f3ae9cba916275, user_id=3db3c6d0b14a497d9aeee6ea7ab09546, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:47.238957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.255953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1155/2305] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:47 2022] PUT /identity/v3/projects/113b0f00d7b04db0b1f3ae9cba916275/users/3db3c6d0b14a497d9aeee6ea7ab09546/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:47.266390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.268425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:47.284064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe8ecd-5a02-4514-ab01-4dea549aff84 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:47.287834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:47.407638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:47.408963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:47.409246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:47.409522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] PATH_INFO: `/v3/projects/e4d56ec9666c41179fa93929abb8bc47` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:47.411386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d56ec9666c41179fa93929abb8bc47)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:47.417249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.493487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d56ec9666c41179fa93929abb8bc47', 'request_id': 'req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.505466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d56ec9666c41179fa93929abb8bc47', 'request_id': 'req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.528983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d56ec9666c41179fa93929abb8bc47', 'request_id': 'req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.552985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] The token cache is being invalidate because project e4d56ec9666c41179fa93929abb8bc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:47.553436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.554946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.558675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1151/2306] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:47 2022] DELETE /identity/v3/projects/e4d56ec9666c41179fa93929abb8bc47 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:47.564028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.565986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:47.582682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbae4-a967-414c-8ba9-004087cdfa8c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:47.668626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.670260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:47.725510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:47.726286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:47.726732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:47.727075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] PATH_INFO: `/v3/users/ac08a2e5d6694f2195cd28d29495cf11` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:47.729805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac08a2e5d6694f2195cd28d29495cf11)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:47.733988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.779819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:47.851961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac08a2e5d6694f2195cd28d29495cf11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.863541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac08a2e5d6694f2195cd28d29495cf11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.863751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bd139-6158-42c0-a5f8-d3ba0fb9bf7e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:47.864300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac08a2e5d6694f2195cd28d29495cf11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.865351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac08a2e5d6694f2195cd28d29495cf11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.870215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:47.871306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:47.872447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:47.872934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] PATH_INFO: `/v3/projects/bfeede09780f4d7f8136c5f06001412f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:47.874928 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfeede09780f4d7f8136c5f06001412f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:47.881176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac08a2e5d6694f2195cd28d29495cf11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:47.881365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:47.938614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1156/2307] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:47 2022] DELETE /identity/v3/users/ac08a2e5d6694f2195cd28d29495cf11 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:47.939548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:47.943423 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:48.058398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:48.074880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfeede09780f4d7f8136c5f06001412f', 'request_id': 'req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.094149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfeede09780f4d7f8136c5f06001412f', 'request_id': 'req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.111027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfeede09780f4d7f8136c5f06001412f', 'request_id': 'req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.127023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] The token cache is being invalidate because project bfeede09780f4d7f8136c5f06001412f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:48.128165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.128857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.131843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1152/2308] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:47 2022] DELETE /identity/v3/projects/bfeede09780f4d7f8136c5f06001412f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:48.136261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:48.139226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:48.203280 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7b16-1c81-472d-8193-3dcb546c33d9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:48.310428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45aaf95-a642-4ae2-8fa2-d40e4b3c781b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:48.323865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:48.325791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:48.338671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:48.340730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] PATH_INFO: `/v3/users/c57caa65e3bf48c9b36d156f5fd5a69a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:48.343967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57caa65e3bf48c9b36d156f5fd5a69a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:48.349929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:48.373389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:48.374043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:48.374323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:48.378261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] PATH_INFO: `/v3/projects/fd7c3947ad0245658d115c7c8b13997e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:48.381851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7c3947ad0245658d115c7c8b13997e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:48.383823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:48.450836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57caa65e3bf48c9b36d156f5fd5a69a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.478532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57caa65e3bf48c9b36d156f5fd5a69a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.479507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57caa65e3bf48c9b36d156f5fd5a69a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.480205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57caa65e3bf48c9b36d156f5fd5a69a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.494626 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0488412c-5fbc-4f0d-8320-5938cc32a21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57caa65e3bf48c9b36d156f5fd5a69a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.548247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1153/2309] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:48 2022] DELETE /identity/v3/users/c57caa65e3bf48c9b36d156f5fd5a69a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:48.571510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7c3947ad0245658d115c7c8b13997e', 'request_id': 'req-441cde16-0e34-462e-a68a-7119b33bca37'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.580524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:48.584205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7c3947ad0245658d115c7c8b13997e', 'request_id': 'req-441cde16-0e34-462e-a68a-7119b33bca37'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.596170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7c3947ad0245658d115c7c8b13997e', 'request_id': 'req-441cde16-0e34-462e-a68a-7119b33bca37'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.619419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] The token cache is being invalidate because project fd7c3947ad0245658d115c7c8b13997e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:48.620169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.623247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-441cde16-0e34-462e-a68a-7119b33bca37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:48.625977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1157/2310] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:47 2022] DELETE /identity/v3/projects/fd7c3947ad0245658d115c7c8b13997e => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:48.629837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:48.630127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:48.630769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:48.670689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:48.672226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:48.792370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:48.836805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1154/2311] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:48.845132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:48.845419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:48.845703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:48.883445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:48.884781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:48.901233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1158/2312] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:48.907728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:48.925374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9300c2-9ca2-4680-9d2f-053ea4ccbaae None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:48.931724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:48.932489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:48.933028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:48.933927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:48.934262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:48.934631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:48.944459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:48.969739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1155/2313] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:48.974168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:48.990762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca291-92cb-4a60-b1ad-d4edda930db8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.001847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.002780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.011545 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.012260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.013249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.024593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.080943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1159/2314] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:48 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:49.084965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.106524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf51ed-45c7-488a-b065-8b0b14f277e1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.175153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.177147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:49.295947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.297748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.298902 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.299595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.302614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.307152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.401344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1156/2315] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:49 2022] POST /identity/v3/users => generated 287 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:49.402196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.426061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aad79-f0e8-4c81-bbb9-a18d014901cb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.430969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6533f-9e6e-4028-82e0-bdc472ee5e21 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.440334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.441038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.441328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.441606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.443502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.446838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.447670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.447956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.448229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] PATH_INFO: `/v3/projects/f93b7f12345c475ba90fe805ef9ca539` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.450178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.451482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93b7f12345c475ba90fe805ef9ca539)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.453447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.472735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1157/2316] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:49.477199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.489962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d49e1-7b47-4f2e-aebd-650202e67d74 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.495691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.497220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.497724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.498181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.499245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.501075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.517992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1158/2317] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:44:49.519740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.539723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b2d8-7ca6-4df7-ac8d-ce1388adaa6e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:49.551725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:49.552919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.553386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.553968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] PATH_INFO: `/v3/projects/ff0f334384454c7daba615e52ce21cd1/users/b0da6e03372642b8b2ee06780cd03714/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.604256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0f334384454c7daba615e52ce21cd1, user_id=b0da6e03372642b8b2ee06780cd03714, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:49.605432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93b7f12345c475ba90fe805ef9ca539', 'request_id': 'req-7c2af630-13db-448a-afa1-8f04ac51ec82'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:49.611903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d53ac2-24dd-4a43-a021-692bc46621c7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:49.615939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93b7f12345c475ba90fe805ef9ca539', 'request_id': 'req-7c2af630-13db-448a-afa1-8f04ac51ec82'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:49.637638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93b7f12345c475ba90fe805ef9ca539', 'request_id': 'req-7c2af630-13db-448a-afa1-8f04ac51ec82'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:49.642325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1159/2318] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:49 2022] PUT /identity/v3/projects/ff0f334384454c7daba615e52ce21cd1/users/b0da6e03372642b8b2ee06780cd03714/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:49.656247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:49.659193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:49.659503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:49.666144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] The token cache is being invalidate because project f93b7f12345c475ba90fe805ef9ca539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:49.666457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:49.667180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:49.668751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1160/2319] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:49 2022] DELETE /identity/v3/projects/f93b7f12345c475ba90fe805ef9ca539 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:49.711514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:49.713348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:49.936513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1160/2320] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:49.977890 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:49.979669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:50.075782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:50.099622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2af630-13db-448a-afa1-8f04ac51ec82 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:50.212258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:50.213722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.214732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.216214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.252385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:50.255544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2de47b-0552-47cf-beb2-138270d25e70 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:50.313626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1161/2321] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:50.400936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:50.402472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:50.417239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eea82-03d0-4210-b98c-44572afe0b8f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:50.423579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:50.424661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.425172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.425733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] PATH_INFO: `/v3/projects/be437018df094c219a6b39210399e1f7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.427639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be437018df094c219a6b39210399e1f7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:50.430085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:50.457016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:50.559861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be437018df094c219a6b39210399e1f7', 'request_id': 'req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:50.570713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be437018df094c219a6b39210399e1f7', 'request_id': 'req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:50.594120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be437018df094c219a6b39210399e1f7', 'request_id': 'req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:50.619034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] The token cache is being invalidate because project be437018df094c219a6b39210399e1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:50.620351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:50.620686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54120c08-f253-4ad0-8ba6-2a096fb6fbe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:50.623493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1161/2322] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:50 2022] DELETE /identity/v3/projects/be437018df094c219a6b39210399e1f7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:50.628620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.629112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.629391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.663470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:50.664683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:50.748604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1162/2323] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:50.758388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.758836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.759185 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.796318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:50.802904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:50.822894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1162/2324] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:50.825385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:50.885754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e839b-1e0c-41e3-8f30-12371e9538fe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:50.907680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:50.909475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.909762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.910037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.911924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:50.912263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:50.918792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:50.960576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:50.961434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1163/2325] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:44:50.977411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917875-f1c1-4428-bc7c-f37444c71d37 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:50.983883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:50.984767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:50.985205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:50.985631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:50.986452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:50.988564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.032504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.034210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1163/2326] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:50 2022] POST /identity/v3/projects => generated 357 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:51.044841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3592d8-b6ac-4405-b474-6dfe847571d4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.127059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.127604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.128197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.128197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.128766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.130741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.156127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1164/2327] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:51 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:51.160142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.171653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88f38-ba0d-4f8e-9592-15aeecaceb8d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.176982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.177602 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.177875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.178136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.179233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.180877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.192814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1164/2328] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:51.197218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.208410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf14e6-c0f9-4369-9ec4-66387caf2ecd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.213246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.213848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.214124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.214394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.215107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.216740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441d4f-256b-4f43-b18e-dad39db26d37 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.233153 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.233573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1165/2329] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:51.243634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e5c30-153a-445d-aeb6-fc008a3f9704 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.256400 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.257675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.258060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.258923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] PATH_INFO: `/v3/projects/03333d81470b4be498d67c4c3a2434a5/users/c88812ef7e274f3c89cc1ba39ebc68de/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.287288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03333d81470b4be498d67c4c3a2434a5, user_id=c88812ef7e274f3c89cc1ba39ebc68de, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.290936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.315482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1165/2330] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:51 2022] PUT /identity/v3/projects/03333d81470b4be498d67c4c3a2434a5/users/c88812ef7e274f3c89cc1ba39ebc68de/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:51.325991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.326275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.326628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.353021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:51.366663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] MFA Rules not processed for user `c88812ef7e274f3c89cc1ba39ebc68de`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:51.368236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:51.467512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1166/2331] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:51.471808 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.584869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c550f5-deaf-46dc-9005-723625301c2e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.591202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.592101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.592526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.592972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.593886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.596180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.632845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1166/2332] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:51 2022] POST /identity/v3/projects => generated 355 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:51.636544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.655693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a79c0d-e5c3-40ac-bd90-d92dcac8cd37 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.771431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.772082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.772368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.772647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.775356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.786515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.844668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1167/2333] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:51 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:51.848812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.864196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2738-d4b0-40ab-8794-fcbe43a92a67 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.878325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.879179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.879534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.880037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.880931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.889671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:51.914337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1167/2334] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:51.916042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:51.931558 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5de2-381c-4bc5-b15d-430960f8c6b3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:51.945086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:51.946294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:51.948993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:51.950231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] PATH_INFO: `/v3/projects/71a0639b4e364073a8a6e794c989cff7/users/41609b2030374ae4ab2011937a491882/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:51.980181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a0639b4e364073a8a6e794c989cff7, user_id=41609b2030374ae4ab2011937a491882, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:51.986136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.028233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1168/2335] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:51 2022] PUT /identity/v3/projects/71a0639b4e364073a8a6e794c989cff7/users/41609b2030374ae4ab2011937a491882/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:52.028678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.041364 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949eec-2ed8-4b7a-979c-476936ebc417 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.052387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:52.133182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.133929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.134210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.134557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.135555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.138570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585eb1b-dc89-40d1-b43b-86e07b3833ec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.157818 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.167524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1168/2336] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:52.176492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d93b0-d71a-4c31-bdbd-12470958c43f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.186066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.187790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.188241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.188713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] PATH_INFO: `/v3/projects/71a0639b4e364073a8a6e794c989cff7/users/41609b2030374ae4ab2011937a491882/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.191074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a0639b4e364073a8a6e794c989cff7, user_id=41609b2030374ae4ab2011937a491882, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.194524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.221606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac3c26-db56-4f2b-83b7-3c9f4c03df68 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.222134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac3c26-db56-4f2b-83b7-3c9f4c03df68 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.222600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac3c26-db56-4f2b-83b7-3c9f4c03df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.225376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1169/2337] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:52 2022] PUT /identity/v3/projects/71a0639b4e364073a8a6e794c989cff7/users/41609b2030374ae4ab2011937a491882/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:52.288886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d4ac3c26-db56-4f2b-83b7-3c9f4c03df68 None None] MFA Rules not processed for user `41609b2030374ae4ab2011937a491882`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:52.290950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d4ac3c26-db56-4f2b-83b7-3c9f4c03df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.459014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1169/2338] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:52.472919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.484297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.551862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.552215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.552487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.608569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] MFA Rules not processed for user `b0da6e03372642b8b2ee06780cd03714`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:52.620148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.623262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3bd69-7a9d-4a4b-b4ec-a6f171f0225e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.628791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.629662 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.630090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.630659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.647663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:52.661005 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.663281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.687998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1170/2339] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:52.745914 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1170/2340] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:52.758874 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.770169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6d71-bfc0-435f-93b4-53196b4a7f23 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.779684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.780565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.781269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.782827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.798374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.799758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d606f-8737-4212-9263-c3367e94038b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.802801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.804429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.819050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1171/2341] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:52.828720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a5f80-23fc-4f1d-b9c7-f74477154553 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.834723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.835563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.836023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.836700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.863467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:52.914819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.916563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.930362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1171/2342] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:54.008086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b37e-fee0-4754-afd6-c0213c107c7f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:54.008638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b37e-fee0-4754-afd6-c0213c107c7f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:54.008638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b37e-fee0-4754-afd6-c0213c107c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:54.038144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-92b3b37e-fee0-4754-afd6-c0213c107c7f None None] MFA Rules not processed for user `3db3c6d0b14a497d9aeee6ea7ab09546`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:54.039432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-92b3b37e-fee0-4754-afd6-c0213c107c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:54.109873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1172/2343] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:54.124191 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:54.125741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:54.209905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b36560-071e-4326-8920-96622d6f5e82 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:54.215534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:54.216141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:54.216425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:54.216704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:54.218455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:54.295743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:54.297743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6b8d7-7f5b-43aa-9635-6a216153bb69 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:54.317373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1172/2344] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:55.572629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:55.573406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:55.573406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:55.640934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] MFA Rules not processed for user `b0da6e03372642b8b2ee06780cd03714`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:55.648887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:55.668061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1173/2345] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:55.945680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2f1e-b1d5-4463-b34c-ae9d5e67737f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:55.946360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2f1e-b1d5-4463-b34c-ae9d5e67737f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:55.946360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2f1e-b1d5-4463-b34c-ae9d5e67737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:55.989930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b90a2f1e-b1d5-4463-b34c-ae9d5e67737f None None] MFA Rules not processed for user `0ca903cc98a24621b7c3be488e3e7c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:55.991169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b90a2f1e-b1d5-4463-b34c-ae9d5e67737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:56.006459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1173/2346] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:44:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:05.594486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=46 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:05.968525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=77 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:20.658371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:20.722261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868b3c8-40a3-4251-a220-a464f0604ceb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:20.726784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:20.727419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:20.727698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:20.727974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] PATH_INFO: `/v3/users/c88812ef7e274f3c89cc1ba39ebc68de` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:20.729099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88812ef7e274f3c89cc1ba39ebc68de)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:20.731081 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:20.782937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88812ef7e274f3c89cc1ba39ebc68de'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:20.789825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88812ef7e274f3c89cc1ba39ebc68de'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:20.790561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88812ef7e274f3c89cc1ba39ebc68de'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:20.791165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88812ef7e274f3c89cc1ba39ebc68de'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:20.798757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88812ef7e274f3c89cc1ba39ebc68de'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:20.817340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1174/2347] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:20 2022] DELETE /identity/v3/users/c88812ef7e274f3c89cc1ba39ebc68de => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:20.822032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:20.822669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:20.823112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:20.969378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:20.970677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:21.031520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1174/2348] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:21.041310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:21.042749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:21.102694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff74de6-c4ab-4dd9-8081-aee988cd3ee1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:21.108214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:21.108834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:21.109112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:21.109385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:21.123373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:21.125007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:21.136126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1175/2349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:21.456758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:21.458506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:21.474491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9e7d2-70dd-4e79-b91b-00ba424e2066 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:21.480420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:21.481506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:21.481971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:21.483757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] PATH_INFO: `/v3/projects/03333d81470b4be498d67c4c3a2434a5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:21.485556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03333d81470b4be498d67c4c3a2434a5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:21.488477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:21.545322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03333d81470b4be498d67c4c3a2434a5', 'request_id': 'req-194582b8-5cd5-469c-b78a-2bb9e3dc6985'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.553201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03333d81470b4be498d67c4c3a2434a5', 'request_id': 'req-194582b8-5cd5-469c-b78a-2bb9e3dc6985'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.559880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03333d81470b4be498d67c4c3a2434a5', 'request_id': 'req-194582b8-5cd5-469c-b78a-2bb9e3dc6985'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.570685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] The token cache is being invalidate because project 03333d81470b4be498d67c4c3a2434a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:21.570970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.571593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.573051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1175/2350] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:21 2022] DELETE /identity/v3/projects/03333d81470b4be498d67c4c3a2434a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:21.577008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:21.586648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa8d01-5228-4206-bb80-dfbc544018d5 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:21.605282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:21.646524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:21.647156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:21.647432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:21.647707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] PATH_INFO: `/v3/users/41609b2030374ae4ab2011937a491882` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:21.648817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41609b2030374ae4ab2011937a491882)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:21.650765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:21.698522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41609b2030374ae4ab2011937a491882'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.705441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41609b2030374ae4ab2011937a491882'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.706101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41609b2030374ae4ab2011937a491882'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.706729 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41609b2030374ae4ab2011937a491882'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.713842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-266ac84d-8ef5-49f1-acfe-130c42052e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41609b2030374ae4ab2011937a491882'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:21.731230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1176/2351] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:21 2022] DELETE /identity/v3/users/41609b2030374ae4ab2011937a491882 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:22.008646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.010804 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:22.024500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194582b8-5cd5-469c-b78a-2bb9e3dc6985 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.086329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.086936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.087198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.087471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] PATH_INFO: `/v3/projects/71a0639b4e364073a8a6e794c989cff7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.088595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a0639b4e364073a8a6e794c989cff7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.090528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.120724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:22.139606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a0639b4e364073a8a6e794c989cff7', 'request_id': 'req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:22.147234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a0639b4e364073a8a6e794c989cff7', 'request_id': 'req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:22.153423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a0639b4e364073a8a6e794c989cff7', 'request_id': 'req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:22.163976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] The token cache is being invalidate because project 71a0639b4e364073a8a6e794c989cff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:22.164264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:22.164883 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c1b8f6bd-285d-4625-8c7b-54bbecaaafaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:22.166325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1176/2352] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:22 2022] DELETE /identity/v3/projects/71a0639b4e364073a8a6e794c989cff7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:22.171706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.171994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.172274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.192648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:22.193853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:22.254560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1177/2353] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:22.262235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.263032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.263307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.283944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:22.285176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:22.293126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1177/2354] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:22.297481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.307812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bfff3-fe2b-4b60-a53f-de3d11c44233 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.313353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.313965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.314242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.314551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.315457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:45:22.315766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.317384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.333340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1178/2355] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:45:22.334465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.344353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b735cb5-0225-4340-9fbb-0f92f8740ceb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.349060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.349658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.349939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.350214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.350882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.352825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.382568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1178/2356] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:22.387015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.398061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54da8-e18c-48e5-9a16-ec7679d1d09b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.432712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:22.475112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.475832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.476177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.476524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.477241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.479334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.506476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1179/2357] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:22.508430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.520714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93c4c2-d013-4bd2-8da1-f6d862ef9d4f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.531268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.532186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.532536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.532836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.533489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.535403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.550071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1179/2358] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:22.551059 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.562557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a311-8c97-47c5-80e3-1e1e509dcdc0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.567399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.568004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.568285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.568560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.569205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.570841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a9388-18a4-4154-80b9-33c35dbf95f8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.582227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1180/2359] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:22.586549 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.597909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3da63-6aec-41ad-8782-2de73c0aa59d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.602511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.603163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.603440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.603712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] PATH_INFO: `/v3/projects/46e7bf673dce43e1bbbfb9e6800352b3/users/c80a6d0e82a24015bd4bb594a0fa43f3/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.618457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e7bf673dce43e1bbbfb9e6800352b3, user_id=c80a6d0e82a24015bd4bb594a0fa43f3, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.621097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.634290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1180/2360] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:22 2022] PUT /identity/v3/projects/46e7bf673dce43e1bbbfb9e6800352b3/users/c80a6d0e82a24015bd4bb594a0fa43f3/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:22.645126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.645443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.645722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.672881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] MFA Rules not processed for user `c80a6d0e82a24015bd4bb594a0fa43f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:22.674131 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:22.737510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1181/2361] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:22.741970 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.802465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72dd4-f650-4bb4-b5ca-f72780284aec admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.808252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.808845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.809123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.809394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.810016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.812501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.842938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1181/2362] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:22.847118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.858397 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14644c65-97df-42d6-b9d2-123cc426fd56 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.916762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.917373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.917654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.917927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.918609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.921097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.942962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1182/2363] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:22 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:45:22.946898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.957338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bedca-2a4e-4d49-859f-31949fad6475 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.962134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.962905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.963199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.963491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:22.964239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:22.965880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:22.976981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1182/2364] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:22.980899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:22.991509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5293-4d5e-4de7-af21-4bf8daedaeae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:22.996168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:22.996824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:22.997190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:22.997377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] PATH_INFO: `/v3/projects/5e722524de17431a970cfe339626d8f9/users/43fabc943b524bd191a89ab722f6552c/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.013655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e722524de17431a970cfe339626d8f9, user_id=43fabc943b524bd191a89ab722f6552c, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:23.016387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:23.029981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1183/2365] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:22 2022] PUT /identity/v3/projects/5e722524de17431a970cfe339626d8f9/users/43fabc943b524bd191a89ab722f6552c/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:23.034476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:23.045408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17b29-a143-489b-8fd7-0c6d444eb078 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:23.066589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:23.109207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:23.109777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.110053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.110324 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.111008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:23.112608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409b249-6513-4cab-9c64-2af54d435770 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:23.122297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1183/2366] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:45:23.127216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:23.132773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:23.139462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d5bfb-9ad0-466d-a743-451c7e3517cb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:23.145353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:23.146011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.146294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.146615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] PATH_INFO: `/v3/projects/5e722524de17431a970cfe339626d8f9/users/43fabc943b524bd191a89ab722f6552c/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.148557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e722524de17431a970cfe339626d8f9, user_id=43fabc943b524bd191a89ab722f6552c, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:23.151306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:23.162756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1184/2367] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:23 2022] PUT /identity/v3/projects/5e722524de17431a970cfe339626d8f9/users/43fabc943b524bd191a89ab722f6552c/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:45:23.174368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.174758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.175128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.204259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] MFA Rules not processed for user `43fabc943b524bd191a89ab722f6552c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:23.205501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:23.274758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1184/2368] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:23.286552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:23.288414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:23.353494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010eacc-b4e2-4e92-ac6c-0bef60c44409 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:23.357826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:23.358436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.358731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.359006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.368313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:23.369670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:23.379322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1185/2369] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:23.458643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:23.468996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d985e0-a263-4949-acaa-c35add4060cb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:23.473577 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:23.474842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.475124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.475699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.538126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:23.539622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cdb06-5919-43de-8e40-1670fad01e8c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:23.550852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1185/2370] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:24.634935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:24.636354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:24.708699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab067-15a3-44a9-b726-4ec7f5b53fff service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:24.713562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:24.714304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:24.714664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:24.714946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] PATH_INFO: `/v3/users/c631fd48c2774f519a56672fb0246ed8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:24.716076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c631fd48c2774f519a56672fb0246ed8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:24.718526 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:24.783541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c631fd48c2774f519a56672fb0246ed8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:24.792357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c631fd48c2774f519a56672fb0246ed8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:24.793430 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c631fd48c2774f519a56672fb0246ed8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:24.794074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c631fd48c2774f519a56672fb0246ed8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:24.804784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c26cd48-ae84-4527-ae44-0a446d272e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c631fd48c2774f519a56672fb0246ed8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:24.848666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:24.849917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:24.850199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:24.855125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1186/2371] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:24 2022] DELETE /identity/v3/users/c631fd48c2774f519a56672fb0246ed8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:45:24.864043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:24.864363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:24.864634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:24.878669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] MFA Rules not processed for user `d8581d46699041d5a4dcb8a43034863e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:24.880158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:24.890075 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:24.891847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:24.897805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:24.950648 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:24.974791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1187/2372] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:24.975714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1186/2373] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:24.985127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:24.986852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:25.079165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9de2b-8a3a-47a7-bb02-c796c3274bb8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:25.083471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:25.084091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.084368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.084635 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.100381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:25.102904 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924f29-8eab-4a0b-8c71-74948c1bd8fb service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:25.117661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1188/2374] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:25.593962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:25.595429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:25.609832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313685-d8fb-48da-af37-e9ff52e7dd87 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:25.625290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:25.625948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.626220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.626513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] PATH_INFO: `/v3/projects/5359f4a3a0ad43bcaf5191876513eb1e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.627627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5359f4a3a0ad43bcaf5191876513eb1e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:25.629592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:25.676690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5359f4a3a0ad43bcaf5191876513eb1e', 'request_id': 'req-187b8cbf-5d21-4fb1-909f-a524baf74796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:25.684272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5359f4a3a0ad43bcaf5191876513eb1e', 'request_id': 'req-187b8cbf-5d21-4fb1-909f-a524baf74796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:25.691551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5359f4a3a0ad43bcaf5191876513eb1e', 'request_id': 'req-187b8cbf-5d21-4fb1-909f-a524baf74796'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:25.703081 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] The token cache is being invalidate because project 5359f4a3a0ad43bcaf5191876513eb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:25.703366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:25.704119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187b8cbf-5d21-4fb1-909f-a524baf74796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:25.705820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1187/2375] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:25 2022] DELETE /identity/v3/projects/5359f4a3a0ad43bcaf5191876513eb1e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:25.710650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.711120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.711227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.731107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:25.732350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:25.795608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1189/2376] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:25.803363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.803781 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.804202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.830200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:25.835586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:25.844371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1188/2377] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:25.849634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:25.861440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfdfff-cbdb-4648-9f2e-c87fae514444 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:25.868148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:25.868493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.868775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.869047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.869689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:45:25.869990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:25.871660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:25.894928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1190/2378] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:45:25.900321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:25.909637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698f0d-a096-43b5-a113-901bf5cde7ed None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:25.914989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:25.915407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:25.915724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:25.915946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:25.916578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:25.918504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:25.950475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1189/2379] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:25 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:45:25.953133 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:25.963933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aea13-fe49-4984-ac5d-f60719df2519 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.008265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:26.024993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.025598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.025878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.026150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.026813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.028800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.050070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1191/2380] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:25 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:26.054636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.074651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c204e-9c80-4dca-91ed-66d404f9d556 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.079308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.080142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.080484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.080827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.081634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.083433 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.095360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1190/2381] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:45:26.099756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.111307 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c9ba8-5f75-4fbc-9ade-37dd4d8c724a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.116495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.117128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.117403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.117684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.118332 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.119994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821139f-fe59-45e4-a377-7400236ea31c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.133795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1192/2382] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:26.139007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.155281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11264e-c648-468a-8299-4db6924b74d1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.160826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.161508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.161800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.162065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] PATH_INFO: `/v3/projects/bb5cdf25d821410b9751d0d4c14b91eb/users/bb2c4ee338c84aca9be9e4b58806da84/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.178988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb5cdf25d821410b9751d0d4c14b91eb, user_id=bb2c4ee338c84aca9be9e4b58806da84, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.181719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.199490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1191/2383] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:26 2022] PUT /identity/v3/projects/bb5cdf25d821410b9751d0d4c14b91eb/users/bb2c4ee338c84aca9be9e4b58806da84/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:26.211742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.212001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.212277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.242094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] MFA Rules not processed for user `bb2c4ee338c84aca9be9e4b58806da84`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:26.243338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:26.328478 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1193/2384] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:26.339924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.342884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:26.413237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1588b-f67b-4e5f-bad4-cafe1ef881b3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.418713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.419474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.420280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.420557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.433366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.435181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fa50-8af9-469b-9738-18efaf89ba81 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.446812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1192/2385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:26.489022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.569836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e01b49-f7f0-4909-b048-ce9c4b032fd6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.574529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.575157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.575433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.576217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] PATH_INFO: `/v3/users/bb2c4ee338c84aca9be9e4b58806da84` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.577348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2c4ee338c84aca9be9e4b58806da84)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.579315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.626516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2c4ee338c84aca9be9e4b58806da84'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:26.630902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:26.634767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2c4ee338c84aca9be9e4b58806da84'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:26.635683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2c4ee338c84aca9be9e4b58806da84'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:26.636637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2c4ee338c84aca9be9e4b58806da84'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:26.644066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2c4ee338c84aca9be9e4b58806da84'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:26.669923 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1194/2386] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:26 2022] DELETE /identity/v3/users/bb2c4ee338c84aca9be9e4b58806da84 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:26.674402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119705-658d-4289-8d2c-ef808beadaad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.674717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119705-658d-4289-8d2c-ef808beadaad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.674992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119705-658d-4289-8d2c-ef808beadaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.694835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-52119705-658d-4289-8d2c-ef808beadaad None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:26.696049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52119705-658d-4289-8d2c-ef808beadaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:26.701410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:26.756824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1193/2387] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:26.766810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:26.768298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:26.828067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19eda2-0ca5-4ef2-98e3-48feb7299492 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:26.831932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:26.832537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:26.832822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:26.833089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:26.843337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:26.844814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7115c1-8808-4383-a314-599356a21302 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:26.854779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1195/2388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:27.174326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52119705-658d-4289-8d2c-ef808beadaad None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.176032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52119705-658d-4289-8d2c-ef808beadaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:27.205354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52119705-658d-4289-8d2c-ef808beadaad None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.210882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.211621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.212030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.212431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] PATH_INFO: `/v3/projects/bb5cdf25d821410b9751d0d4c14b91eb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.213723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb5cdf25d821410b9751d0d4c14b91eb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.216197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.271223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb5cdf25d821410b9751d0d4c14b91eb', 'request_id': 'req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:27.278926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb5cdf25d821410b9751d0d4c14b91eb', 'request_id': 'req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:27.286093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb5cdf25d821410b9751d0d4c14b91eb', 'request_id': 'req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:27.297297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] The token cache is being invalidate because project bb5cdf25d821410b9751d0d4c14b91eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:27.297582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:27.298230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a5f8f6c-1620-4c68-a0f4-d0c1b71d534b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:27.299980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1194/2389] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:27 2022] DELETE /identity/v3/projects/bb5cdf25d821410b9751d0d4c14b91eb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:27.305212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.305530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.305780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.328152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:27.329384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:27.398079 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1196/2390] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:27.405806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.406134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.406351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.429850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:27.431099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:27.454699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1195/2391] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:27.454805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.464741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40513528-b282-4c17-a3cf-52a1577d8ee5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.469993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.470786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.471327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.471605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.472345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:45:27.472759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.474665 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.494978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1197/2392] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:45:27.496130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.509040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e307f-8ae4-42af-af3a-2bbc99ca2ed3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.514177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.514835 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.515169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.515555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.516090 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.518068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.558504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1196/2393] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:27 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:45:27.561937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.572982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19bf3-6207-4635-b929-9fcbc6fc3a22 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.698661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.699495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.700437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.700947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.701868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.705065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.715991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:27.739472 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1198/2394] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:27 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:27.743307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.766466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdaf19-e2be-4ce1-9bb3-e922f9b5277f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.786640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.786640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.786640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.786640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.787180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.795310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.817162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1197/2395] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:27.820961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.840836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b246-e8b6-4ccb-9f7b-bb110bafdb78 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.851606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.852255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.852744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.852882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.853518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.855173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801af89-744a-46c6-af4d-aa8d4d02089b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.880001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1199/2396] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:27.884282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:27.897958 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae28316-25fb-4b51-b73b-ac990c197a58 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:27.903089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:27.904528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.904528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.904733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] PATH_INFO: `/v3/projects/d253f96f738a49b2a561f5d91987215e/users/ef56c51d9fdd4543b67ca27d6d87cde2/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.922468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d253f96f738a49b2a561f5d91987215e, user_id=ef56c51d9fdd4543b67ca27d6d87cde2, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:27.925125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:27.946537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1198/2397] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:27 2022] PUT /identity/v3/projects/d253f96f738a49b2a561f5d91987215e/users/ef56c51d9fdd4543b67ca27d6d87cde2/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:27.956698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69377f-7091-41a5-916e-deb6fedc6337 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:27.957207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69377f-7091-41a5-916e-deb6fedc6337 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:27.957646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69377f-7091-41a5-916e-deb6fedc6337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:27.996919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7a69377f-7091-41a5-916e-deb6fedc6337 None None] MFA Rules not processed for user `ef56c51d9fdd4543b67ca27d6d87cde2`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:27.998597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7a69377f-7091-41a5-916e-deb6fedc6337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:28.088520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1200/2398] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:28.100584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:28.102450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:28.170059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:28.205620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eae8b-832c-4477-9aff-c6d1965e4b84 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:28.210653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:28.211485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:28.211821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:28.212161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:28.225038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:28.226631 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:28.241672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1199/2399] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:33.335302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9548-8b67-4eff-96f5-aa5d5656df92 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:33.336431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9548-8b67-4eff-96f5-aa5d5656df92 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:33.337402 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9548-8b67-4eff-96f5-aa5d5656df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:33.359991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-beab9548-8b67-4eff-96f5-aa5d5656df92 None None] MFA Rules not processed for user `b0da6e03372642b8b2ee06780cd03714`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:33.361205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-beab9548-8b67-4eff-96f5-aa5d5656df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:33.420825 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1201/2400] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:45:38.236593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=12 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:43.413077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=3,keystone:SELECT=64 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:45.326675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:45.328148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:45.415348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e826b7-ebc1-447b-970c-a23f6d2250a2 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:45.424196 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:45.425404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:45.425841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:45.426087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] PATH_INFO: `/v3/users/c80a6d0e82a24015bd4bb594a0fa43f3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:45.445564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80a6d0e82a24015bd4bb594a0fa43f3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:45.447698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:45.521574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80a6d0e82a24015bd4bb594a0fa43f3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:45.534992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80a6d0e82a24015bd4bb594a0fa43f3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:45.535850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80a6d0e82a24015bd4bb594a0fa43f3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:45.536848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80a6d0e82a24015bd4bb594a0fa43f3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:45.560997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80a6d0e82a24015bd4bb594a0fa43f3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:45.592893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1200/2401] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:45 2022] DELETE /identity/v3/users/c80a6d0e82a24015bd4bb594a0fa43f3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:45.594629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:45.595090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:45.595475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:45.625510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:45.627980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:45.707775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1202/2402] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:45:45.717901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:45.719465 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:45.781176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e971ef-22a7-4448-ad25-5e5a367adc62 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:45.784880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:45.785594 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:45.785924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:45.786263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:45.796136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:45.797559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:45.808343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1201/2403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:46.344004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:46.345467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:46.357315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f19581-f63d-4212-b52a-e376c97c89ba None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:46.362156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:46.362820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:46.363099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:46.363372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] PATH_INFO: `/v3/projects/46e7bf673dce43e1bbbfb9e6800352b3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:46.364479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e7bf673dce43e1bbbfb9e6800352b3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:46.366492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:46.419582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e7bf673dce43e1bbbfb9e6800352b3', 'request_id': 'req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.425748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e7bf673dce43e1bbbfb9e6800352b3', 'request_id': 'req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.433072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e7bf673dce43e1bbbfb9e6800352b3', 'request_id': 'req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.445975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] The token cache is being invalidate because project 46e7bf673dce43e1bbbfb9e6800352b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:46.446283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.447064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.448546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1203/2404] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:46 2022] DELETE /identity/v3/projects/46e7bf673dce43e1bbbfb9e6800352b3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:46.452997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:46.462545 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1239cc-f727-4fec-8e6d-8c684aa91060 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:46.481848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:46.521052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:46.521677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:46.521960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:46.522235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] PATH_INFO: `/v3/users/43fabc943b524bd191a89ab722f6552c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:46.523427 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fabc943b524bd191a89ab722f6552c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:46.525425 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:46.570369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fabc943b524bd191a89ab722f6552c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.577771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fabc943b524bd191a89ab722f6552c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.578490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fabc943b524bd191a89ab722f6552c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.579097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fabc943b524bd191a89ab722f6552c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.586332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c19590c3-e7ba-475a-bc08-6ba94ffc99b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fabc943b524bd191a89ab722f6552c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:46.606148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1202/2405] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:46 2022] DELETE /identity/v3/users/43fabc943b524bd191a89ab722f6552c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:46.871454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:46.873518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:46.889303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3c8c0-39d9-4714-83ca-6c4fcc14e2ba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:46.948514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:46.949124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:46.949405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:46.949677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] PATH_INFO: `/v3/projects/5e722524de17431a970cfe339626d8f9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:46.950839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e722524de17431a970cfe339626d8f9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:46.952804 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:46.984137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:47.002926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e722524de17431a970cfe339626d8f9', 'request_id': 'req-782af2cd-0128-4682-af46-1de99cbe9830'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:47.008428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e722524de17431a970cfe339626d8f9', 'request_id': 'req-782af2cd-0128-4682-af46-1de99cbe9830'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:47.016213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e722524de17431a970cfe339626d8f9', 'request_id': 'req-782af2cd-0128-4682-af46-1de99cbe9830'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:47.028012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] The token cache is being invalidate because project 5e722524de17431a970cfe339626d8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:45:47.028602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:47.028981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-782af2cd-0128-4682-af46-1de99cbe9830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:47.030659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1204/2406] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:45:46 2022] DELETE /identity/v3/projects/5e722524de17431a970cfe339626d8f9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:47.037402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.037402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.037402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.056732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:47.058168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:47.115037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1203/2407] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:47.123428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.123716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.123992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.143467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:47.144660 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:47.152455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1205/2408] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:47.156699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.166130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61acf1d-2bf9-42d0-8af3-fe7e94c5f560 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.171311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.172016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.172296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.172570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.173788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:45:47.173788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.175130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.188519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1204/2409] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:45:47.192723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.203243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb4b0a-7997-467b-a4f7-69b9d873583d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.208359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.209039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.209411 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.209758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.210508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.212488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.246311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1206/2410] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:47.250814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.261392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34750a6-3d51-43e9-b598-fdeef7556ce9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.288006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:47.334514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.335135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.337766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.338085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.338770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.341432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.363259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1205/2411] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/users => generated 305 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:47.367349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.377781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf07813-4d92-4823-bf21-839d6d5750bc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.382537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.383152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.383425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.383701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.384350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.385957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.396408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1207/2412] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:45:47.400171 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.410980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25bcb38-03db-4688-9586-3f7b71ecf708 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.427966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.428803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.429083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.429353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.430217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.431873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ea9bb-924b-406a-ba00-b071e4095817 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.445887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1206/2413] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:45:47.450026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.460151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187f00-837b-4a2e-ad0d-161f8fbac59c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.464760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.465945 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.466224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.466536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] PATH_INFO: `/v3/projects/7a83a4970dfb4c6c90e34a19f597b94f/users/934fda33d3a84add911720d0be6b76cf/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.482219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a83a4970dfb4c6c90e34a19f597b94f, user_id=934fda33d3a84add911720d0be6b76cf, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.484897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.498595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1208/2414] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:47 2022] PUT /identity/v3/projects/7a83a4970dfb4c6c90e34a19f597b94f/users/934fda33d3a84add911720d0be6b76cf/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:47.509735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.510232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.510346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.541501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] MFA Rules not processed for user `934fda33d3a84add911720d0be6b76cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:47.542957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:47.613104 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1207/2415] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:47.617395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.679032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98543a36-be67-4684-9a18-a3aa676c0a9e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.684132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.684816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.685098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.685374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.686016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.687979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.716473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1209/2416] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/projects => generated 407 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:47.720493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.731411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775011-75e7-4ec9-ae5c-c1ea4b428603 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.785863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.786491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.786769 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.787034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.787656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.789585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.810799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1208/2417] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:45:47 2022] POST /identity/v3/users => generated 310 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:45:47.814772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.825761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea88f0-9f7f-46fb-a16b-ef9cda90bf0b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.831151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.831811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.832089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.832351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.832994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.834656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.845872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1210/2418] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:45:47.850021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.859851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f145c8c-11bf-40be-8bae-bd25b073abed admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.864415 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.865057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.865340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.865615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] PATH_INFO: `/v3/projects/237cace00ce145ffa251a8e03fd8380b/users/30ae4c11309943f5ac00c1c796210cc8/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.880089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237cace00ce145ffa251a8e03fd8380b, user_id=30ae4c11309943f5ac00c1c796210cc8, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.882800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.896280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1209/2419] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:47 2022] PUT /identity/v3/projects/237cace00ce145ffa251a8e03fd8380b/users/30ae4c11309943f5ac00c1c796210cc8/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:47.900427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.911739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25c8e-841f-4d48-b9e1-a16a270ff001 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:47.930971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:47.970819 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:47.971353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:47.971637 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:47.971913 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:47.972573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:47.974172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0c2b4-8f38-4ef3-8ff0-a48b6b1abf05 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:47.984108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1211/2420] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:45:47.988391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:47.993863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:48.001476 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a9f5-5b6e-4d28-95d8-342b0915abf2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:48.006446 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:48.007101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:48.007384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:48.007664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] PATH_INFO: `/v3/projects/237cace00ce145ffa251a8e03fd8380b/users/30ae4c11309943f5ac00c1c796210cc8/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:48.009564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237cace00ce145ffa251a8e03fd8380b, user_id=30ae4c11309943f5ac00c1c796210cc8, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:48.012846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:48.023040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1210/2421] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:45:47 2022] PUT /identity/v3/projects/237cace00ce145ffa251a8e03fd8380b/users/30ae4c11309943f5ac00c1c796210cc8/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:48.033089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:48.033384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:48.033661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:48.063316 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] MFA Rules not processed for user `30ae4c11309943f5ac00c1c796210cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:48.064527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:48.131414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1212/2422] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:48.142947 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:48.144659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:48.201825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563275c1-3e20-4e92-9830-487148eae9f3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:48.205525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:48.206117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:48.206396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:48.206694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:48.215605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:48.217087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:48.226954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1211/2423] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:48.379338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:48.388866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f6750-da6c-4476-bdf5-66c7858de1c4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:48.392719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:48.393325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:48.393606 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:48.393882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:48.452706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:48.454142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85d40c-bb0d-459e-8755-8e85991da114 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:48.463519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1213/2424] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:58.222608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:58.459869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=80 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:59.449997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:59.451444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:59.522856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f58ab-d0f0-4769-b179-db21cd47b2c4 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:59.528544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:59.529284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:59.529564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:59.529942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] PATH_INFO: `/v3/users/ef56c51d9fdd4543b67ca27d6d87cde2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:59.530986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef56c51d9fdd4543b67ca27d6d87cde2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:59.532934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:59.581256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef56c51d9fdd4543b67ca27d6d87cde2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:59.588688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef56c51d9fdd4543b67ca27d6d87cde2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:59.589550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef56c51d9fdd4543b67ca27d6d87cde2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:59.590952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef56c51d9fdd4543b67ca27d6d87cde2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:59.598925 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef56c51d9fdd4543b67ca27d6d87cde2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:45:59.617507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1212/2425] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:45:59 2022] DELETE /identity/v3/users/ef56c51d9fdd4543b67ca27d6d87cde2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:45:59.622204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:59.622750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:59.622867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:59.643232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:59.644663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:59.705602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1214/2426] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:59.716216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:59.717967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:59.775791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce781f-57e6-4baa-858d-80dc4ebca94c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:59.780106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:59.780711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:59.780988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:59.781260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:59.790567 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:59.792150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:59.803789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1213/2427] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:00.090368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.092884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.109909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848a77f-3624-4251-a56f-89b16bfeee64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.118278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.120537 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.121590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.122209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] PATH_INFO: `/v3/projects/d253f96f738a49b2a561f5d91987215e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.123676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d253f96f738a49b2a561f5d91987215e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.125839 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.158340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.160306 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.174999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a225-747a-4a4a-929a-78a8f7903c38 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.190094 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd253f96f738a49b2a561f5d91987215e', 'request_id': 'req-a7e05ae8-cff7-4144-a76f-cf7937abd309'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.196586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd253f96f738a49b2a561f5d91987215e', 'request_id': 'req-a7e05ae8-cff7-4144-a76f-cf7937abd309'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.201521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:00.208045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd253f96f738a49b2a561f5d91987215e', 'request_id': 'req-a7e05ae8-cff7-4144-a76f-cf7937abd309'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.222391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] The token cache is being invalidate because project d253f96f738a49b2a561f5d91987215e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:46:00.222929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.223651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7e05ae8-cff7-4144-a76f-cf7937abd309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.225479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1215/2428] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:46:00 2022] DELETE /identity/v3/projects/d253f96f738a49b2a561f5d91987215e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:00.230455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097ecc-ba58-4511-97c8-de7f707fdc42 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.230760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097ecc-ba58-4511-97c8-de7f707fdc42 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.231021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097ecc-ba58-4511-97c8-de7f707fdc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.253017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3c097ecc-ba58-4511-97c8-de7f707fdc42 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:00.254243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c097ecc-ba58-4511-97c8-de7f707fdc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.260383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.262142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.262778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.263293 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] PATH_INFO: `/v3/users/934fda33d3a84add911720d0be6b76cf` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.267345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934fda33d3a84add911720d0be6b76cf)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.269820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.270078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1216/2429] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:46:00.271509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.271796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.272072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.298871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:00.300098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.314002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1217/2430] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:00.318303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.328959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef96c08-024a-478b-b388-d411f7cf0553 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.334871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.335652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.335934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.336209 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.336871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:46:00.337228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.338898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.355979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1218/2431] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:46:00.366156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934fda33d3a84add911720d0be6b76cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.366441 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.376931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934fda33d3a84add911720d0be6b76cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.377747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934fda33d3a84add911720d0be6b76cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.377959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec227-c3f2-43d4-b254-dc87e5f9a77b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.379197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934fda33d3a84add911720d0be6b76cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.385636 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.386232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.386625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.386954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.387651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.389919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.412021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9431102-ab77-4002-9a86-0a70fdb2c4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934fda33d3a84add911720d0be6b76cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:00.440898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1219/2432] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:46:00.447179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.448770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.458627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1214/2433] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:46:00 2022] DELETE /identity/v3/users/934fda33d3a84add911720d0be6b76cf => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:46:00.463248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.466167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.469197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.499917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:00.506122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:00.507919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.536712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2c5f-a360-4e34-9b86-7f42608051b9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.610437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.610437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.610764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1215/2434] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:00.613790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.614285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.615223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.617174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.620989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.623302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.644417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1220/2435] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:00.647987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.660428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d458-8d4c-48fd-85ae-940cb499f4f1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.665955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.666497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.666786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.667183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.668368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.670041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.675708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:00.682634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1221/2436] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:46:00.686404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.691713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55b604-2233-4932-b743-fbb87b1191ba None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.696964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.697342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279b81-9afe-4f9a-a4a2-df8ccd8d82f2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.697716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.698060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.698502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.702438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.703064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.703347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.704329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.704450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.706046 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.709600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.711297 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870de2c-a81a-4305-b5da-9856a8073d5e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.722463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1222/2437] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:46:00.722662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1216/2438] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:46:00.723705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.736661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69649-34cc-4d35-b06d-4348188233a1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:00.744282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:00.744966 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.745254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.745525 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] PATH_INFO: `/v3/projects/bcffff710dd74209b30a1d02fb2bec46/users/88f34e4fc44e432d9e994ae5dcdbd1ae/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.762466 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcffff710dd74209b30a1d02fb2bec46, user_id=88f34e4fc44e432d9e994ae5dcdbd1ae, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:00.765011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:00.783197 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1223/2439] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:00 2022] PUT /identity/v3/projects/bcffff710dd74209b30a1d02fb2bec46/users/88f34e4fc44e432d9e994ae5dcdbd1ae/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:00.788508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:00.788824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:00.789139 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:00.818225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:00.819628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.910979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1217/2440] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:00.919719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:00.922405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:00.997504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705de06-807f-4c7b-b473-f76f95aa0906 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:01.002852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:01.003471 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:01.003829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:01.004064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:01.015052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:01.017040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:01.027150 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1224/2441] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:01.053811 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:01.055408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:01.070445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba6713-e67d-4c19-89ae-3ea00dcc72a0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:01.076968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:01.077692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:01.078043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:01.078406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] PATH_INFO: `/v3/projects/7a83a4970dfb4c6c90e34a19f597b94f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:01.079698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a83a4970dfb4c6c90e34a19f597b94f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:01.081694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:01.147457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a83a4970dfb4c6c90e34a19f597b94f', 'request_id': 'req-2486180a-b5e8-4155-bce7-47bd1590ccaf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.156739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a83a4970dfb4c6c90e34a19f597b94f', 'request_id': 'req-2486180a-b5e8-4155-bce7-47bd1590ccaf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.164409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a83a4970dfb4c6c90e34a19f597b94f', 'request_id': 'req-2486180a-b5e8-4155-bce7-47bd1590ccaf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.177201 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] The token cache is being invalidate because project 7a83a4970dfb4c6c90e34a19f597b94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:46:01.177522 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.178204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.179823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1218/2442] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:46:01 2022] DELETE /identity/v3/projects/7a83a4970dfb4c6c90e34a19f597b94f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:01.185315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:01.196118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0892b77-ab00-4051-b85b-2d35e932a817 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:01.237347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:01.280165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:01.280887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:01.281175 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:01.281453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] PATH_INFO: `/v3/users/30ae4c11309943f5ac00c1c796210cc8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:01.282638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ae4c11309943f5ac00c1c796210cc8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:01.284581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:01.341965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ae4c11309943f5ac00c1c796210cc8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.354921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ae4c11309943f5ac00c1c796210cc8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.356899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ae4c11309943f5ac00c1c796210cc8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.357549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ae4c11309943f5ac00c1c796210cc8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.372564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5f4bf140-5dda-40c7-92a2-42be995a62f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ae4c11309943f5ac00c1c796210cc8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.396127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1225/2443] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:46:01 2022] DELETE /identity/v3/users/30ae4c11309943f5ac00c1c796210cc8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:46:01.748473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:01.750021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:01.779314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486180a-b5e8-4155-bce7-47bd1590ccaf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:01.849084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:01.849731 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:01.850024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:01.850881 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] PATH_INFO: `/v3/projects/237cace00ce145ffa251a8e03fd8380b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:01.852094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=237cace00ce145ffa251a8e03fd8380b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:01.854678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:01.867580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:01.922858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237cace00ce145ffa251a8e03fd8380b', 'request_id': 'req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.934784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237cace00ce145ffa251a8e03fd8380b', 'request_id': 'req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.943922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237cace00ce145ffa251a8e03fd8380b', 'request_id': 'req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.960174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] The token cache is being invalidate because project 237cace00ce145ffa251a8e03fd8380b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:46:01.960516 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.961218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45da32d7-fcad-42e9-a79e-2b7a4acce2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:01.964021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1219/2444] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:46:01 2022] DELETE /identity/v3/projects/237cace00ce145ffa251a8e03fd8380b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:46:01.969845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a542-9775-47ee-8c85-a39ccdd1fd2e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:01.970958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a542-9775-47ee-8c85-a39ccdd1fd2e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:01.971327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a542-9775-47ee-8c85-a39ccdd1fd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:01.994531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a4f1a542-9775-47ee-8c85-a39ccdd1fd2e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:01.996949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a4f1a542-9775-47ee-8c85-a39ccdd1fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.012264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9a4f-7215-4ee1-821b-e9374f7f6185 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.013260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9a4f-7215-4ee1-821b-e9374f7f6185 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.014328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9a4f-7215-4ee1-821b-e9374f7f6185 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.017300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1220/2445] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 15:46:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:46:02.029734 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.030635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.032024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.054025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:02.055785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.078063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1226/2446] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:02.088955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.089415 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.090367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.113611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:02.115054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.136388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1227/2447] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:02.137870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.154117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94821-96e8-4139-b49b-60b14998b53c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.160060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.160060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.160417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.160678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.161336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:46:02.161647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.163432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.167790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1221/2448] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 15:46:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:46:02.175957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.187969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122504-d0c0-45cd-b3a6-c3da0fbcc263 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.192761 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1228/2449] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:46:02.194386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.194633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.195055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.195331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.195598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.198009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.206775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef3a70-1d12-46b9-8697-ef2231416a20 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.212529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.213138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.213417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.213689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.214328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.216547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.216974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.219583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.236656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1222/2450] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:46:02.278846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1229/2451] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:46:02 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:02.281234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.292484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f327dd-4c23-4c14-9376-e83957a2c5c5 service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.367010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.367973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.368657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.368937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.369652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.372455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.414605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1223/2452] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:46:02 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:46:02.415810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.432938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227a3c-6f1e-461e-8dae-9e476a8b1fc6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.441865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.442728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.442872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.443223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.443907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.445679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.447978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:02.461908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1230/2453] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:02.465645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.478601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09b6ae-d0b3-4b72-90dd-81387cd65387 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.487535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.488521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.489252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.489720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.490694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.492786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1d7f-6d1c-4d55-a605-ff829319e74d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.509355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:02.518447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1224/2454] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:02.520205 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.533207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b6d1c-a493-49f6-83f0-cef80cd48887 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.538783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.539692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.540028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.540302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] PATH_INFO: `/v3/projects/77179790ce7246ef8a3c6b6a93666955/users/f66ba17cc95649c6be8d6940e59df586/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.560906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77179790ce7246ef8a3c6b6a93666955, user_id=f66ba17cc95649c6be8d6940e59df586, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.566008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.592109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1231/2455] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:02 2022] PUT /identity/v3/projects/77179790ce7246ef8a3c6b6a93666955/users/f66ba17cc95649c6be8d6940e59df586/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:02.603163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.603456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.603736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.651109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] MFA Rules not processed for user `f66ba17cc95649c6be8d6940e59df586`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:02.652420 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.756509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1225/2456] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:02.768058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:02.770834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:02.873803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb176-c308-4aaf-a70b-39d2bd6f98d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:02.878782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:02.880171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:02.880465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:02.880736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:02.897067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:02.901714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a8212-f5b3-4342-8e6e-4c19b0abe279 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:02.915868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1232/2457] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:02.944921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:03.064829 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d9da-644f-4359-9493-401589dfc7fa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:03.073682 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:03.075143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:03.075504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:03.075782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] PATH_INFO: `/v3/users/f66ba17cc95649c6be8d6940e59df586` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:03.077414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66ba17cc95649c6be8d6940e59df586)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:03.079967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:03.172210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66ba17cc95649c6be8d6940e59df586'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.197713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66ba17cc95649c6be8d6940e59df586'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.199997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66ba17cc95649c6be8d6940e59df586'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.200752 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66ba17cc95649c6be8d6940e59df586'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.212351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66ba17cc95649c6be8d6940e59df586'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.247230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1226/2458] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:46:02 2022] DELETE /identity/v3/users/f66ba17cc95649c6be8d6940e59df586 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:03.248006 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:03.249004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:03.249322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:03.274785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:03.275912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:03.358991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1233/2459] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:03.369537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:03.372269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:03.386008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:03.512679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda238f-614a-4ac7-bddd-68f094bd3b00 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:03.530896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:03.531618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:03.532047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:03.532393 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:03.544132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:03.545839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ff53b-76a0-4aa0-972a-b5839f717d31 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:03.558035 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1227/2460] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:03.860803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:03.862286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:03.878101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5edb-4e1a-465b-8f04-9e097213380d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:03.885486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:03.887250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:03.888155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:03.888710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] PATH_INFO: `/v3/projects/77179790ce7246ef8a3c6b6a93666955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:03.891960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] RBAC: Authorizing `identity:delete_project(project_id=77179790ce7246ef8a3c6b6a93666955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:03.894080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:03.961995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77179790ce7246ef8a3c6b6a93666955', 'request_id': 'req-798ea595-3122-445b-83aa-1ae65675465c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.969628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77179790ce7246ef8a3c6b6a93666955', 'request_id': 'req-798ea595-3122-445b-83aa-1ae65675465c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.980445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77179790ce7246ef8a3c6b6a93666955', 'request_id': 'req-798ea595-3122-445b-83aa-1ae65675465c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:03.992948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:04.000821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] The token cache is being invalidate because project 77179790ce7246ef8a3c6b6a93666955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:46:04.001269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:04.002052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-798ea595-3122-445b-83aa-1ae65675465c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:04.005047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1234/2461] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:46:03 2022] DELETE /identity/v3/projects/77179790ce7246ef8a3c6b6a93666955 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:04.010105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.010622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.010717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.061800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:04.063452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:04.143266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1228/2462] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:04.146258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.146506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.146852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.176128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:04.176995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:04.189250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1235/2463] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:04.192986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.203965 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4f98-ece6-44ff-ac31-1641aa22f6c2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.210922 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.211880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.212325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.212599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.213943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:46:04.213943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.215426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.233636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1229/2464] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:46:04.238192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.248655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fde33-ea00-4b32-b693-bd58aeeee07f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.267479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.269084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.270156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.270614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.271642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.273570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.336120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1236/2465] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:46:04 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:04.337124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.349336 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae3bf0-58d5-4b35-9530-2a0c4ea4b974 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.441326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.442533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.443340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.443598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.444542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.447518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.489454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1230/2466] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:46:04 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:04.490361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.503646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6357-8e1a-4ac8-92d5-0d52be132144 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.508785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.509984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.510263 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.510618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.511548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.512955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.526117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1237/2467] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:04.527729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.544270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a04bc-bf2b-4bc7-8940-537d006b9561 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.548991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.549595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.550026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.550302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.551040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.552654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc2bae-b017-4336-be62-185ec6f0b714 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.564526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1231/2468] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:04.568566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.578759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec5a7-547c-41b3-8654-5f542480c190 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.583610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.584312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.584588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.584860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] PATH_INFO: `/v3/projects/c34fc4f1109d4982882747f0f92477cb/users/0f6470f8e28347fbb6259000136b140b/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.607106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34fc4f1109d4982882747f0f92477cb, user_id=0f6470f8e28347fbb6259000136b140b, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.610727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.630655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1238/2469] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:04 2022] PUT /identity/v3/projects/c34fc4f1109d4982882747f0f92477cb/users/0f6470f8e28347fbb6259000136b140b/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:04.642624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67626-6f75-4367-9cdb-9cdcd36d573c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.643188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67626-6f75-4367-9cdb-9cdcd36d573c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.643985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67626-6f75-4367-9cdb-9cdcd36d573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.682862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-40f67626-6f75-4367-9cdb-9cdcd36d573c None None] MFA Rules not processed for user `0f6470f8e28347fbb6259000136b140b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:04.684342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-40f67626-6f75-4367-9cdb-9cdcd36d573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:04.691612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:04.772388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1232/2470] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:04.786012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:04.788035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:04.866578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af09f52-cefd-4928-9b89-4811c9035b1d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:04.870624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:04.871692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:04.871692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:04.871907 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:04.883451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:04.885222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:04.905676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1239/2471] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:06.740023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:06.740553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:06.740553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:06.771128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] MFA Rules not processed for user `88f34e4fc44e432d9e994ae5dcdbd1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:06.774450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:06.857109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1233/2472] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:06.861717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:06.863202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:06.972106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb912c-fe2e-4d13-8eb0-fdba7f9f01a1 service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:06.978682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:06.979309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:06.979755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:06.980744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:06.981582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:06.983789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.043866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1240/2473] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:46:06 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:07.048086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:07.059423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3963d-1ca6-4490-94cc-8eff2061d7df None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:07.144904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:07.145535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:07.145819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:07.146094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:07.147141 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:07.149512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.186473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1234/2474] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:46:07 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:07.191431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:07.207653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5d4c-ec69-42c5-af46-d9df76c9b3d6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:07.212425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:07.213048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:07.213324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:07.213597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:07.214251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:07.217177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.228580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1241/2475] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:07.235569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:07.249988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4545b1-2f2c-4bab-9dde-dacdeefe7d92 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:07.266236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:07.267024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:07.267310 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:07.267585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] PATH_INFO: `/v3/projects/48716a5b014047bfbef18ebab68e5cd4/users/7ed1eefc2212459fb0d0935542e9e3f4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:07.284818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48716a5b014047bfbef18ebab68e5cd4, user_id=7ed1eefc2212459fb0d0935542e9e3f4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:07.287546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.306821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1235/2476] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:07 2022] PUT /identity/v3/projects/48716a5b014047bfbef18ebab68e5cd4/users/7ed1eefc2212459fb0d0935542e9e3f4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:07.309605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:07.322956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5344c24-735c-40ae-a513-4265fd2b34f4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:07.335643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:07.396641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:07.397409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:07.397532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:07.397807 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:07.398959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:07.400184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6761b-671e-4969-8e1c-be5e5b053e3c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.422929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1242/2477] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:07.427107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:07.432878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:07.439087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b561-0581-40a7-b197-8cf374b3c4b1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:07.445314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:07.446209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:07.446996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:07.447463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] PATH_INFO: `/v3/projects/48716a5b014047bfbef18ebab68e5cd4/users/7ed1eefc2212459fb0d0935542e9e3f4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:07.450770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48716a5b014047bfbef18ebab68e5cd4, user_id=7ed1eefc2212459fb0d0935542e9e3f4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:07.453434 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:07.470661 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1236/2478] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:07 2022] PUT /identity/v3/projects/48716a5b014047bfbef18ebab68e5cd4/users/7ed1eefc2212459fb0d0935542e9e3f4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:12.782946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:12.783484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:12.783484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:12.817566 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] MFA Rules not processed for user `7ed1eefc2212459fb0d0935542e9e3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:12.818852 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:12.898485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1243/2479] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:12.912231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:12.914526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:13.010200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958e3f-650f-4a21-9490-937e34788e20 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:13.021210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:13.026057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:13.026057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:13.026057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:13.092680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:13.094280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fda1c-ec76-47bb-a31d-38aa8c534f2a service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:13.105274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1237/2480] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:14.099388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:14.103920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:14.190639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f25b4-2bcd-4cb2-9db8-39f2e877ce19 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:14.196909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:14.197765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:14.198200 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:14.199204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:14.201443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:14.262643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:14.284619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:14.286820 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:14.302693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1244/2481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:46:23.099487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:24.295048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:25.654390 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d9447-bfac-4b8e-83ad-ee3daf3ef2df None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:25.654907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d9447-bfac-4b8e-83ad-ee3daf3ef2df None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:25.655000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d9447-bfac-4b8e-83ad-ee3daf3ef2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:25.674160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c1d9447-bfac-4b8e-83ad-ee3daf3ef2df None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:25.675396 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c1d9447-bfac-4b8e-83ad-ee3daf3ef2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:25.732265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1238/2482] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:25.776234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:25.777867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:25.846405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ef209-fa70-415e-95f4-d239a38276d9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:25.851587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:25.852229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:25.852512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:25.852791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:25.861962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:25.863579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:25.872248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1245/2483] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:46:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:46:35.725264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=25 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:35.868362 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=32 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:47.668759 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cced7-60fc-4295-9b7d-d08abb5b667a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:47.669289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cced7-60fc-4295-9b7d-d08abb5b667a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:47.669398 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cced7-60fc-4295-9b7d-d08abb5b667a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:47.688635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b95cced7-60fc-4295-9b7d-d08abb5b667a None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:47.689852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b95cced7-60fc-4295-9b7d-d08abb5b667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:47.696973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1239/2484] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 15:46:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:47.710440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:47.721499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be0af-d621-41e7-a956-7110cb2f1b00 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:47.726653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:47.727480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:47.727619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:47.727786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:47.735749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:47.737325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28732f1e-edb6-4da0-b62a-04fc7370b672 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:47.745256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1246/2485] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:46:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:46:49.720004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:49.720668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:49.720668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:49.722004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1240/2486] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:46:49.752083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:49.752576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:49.752702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:49.774870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] MFA Rules not processed for user `dd633df8700344a19b2de33afb3098ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:49.776554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:49.797583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:49.799330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:49.847192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1247/2487] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:46:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:46:49.858085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:49.859519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:49.871954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d30f6-fdb9-4833-8694-e5fae7dc2abc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:49.873610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f863ebb-7522-4c24-b1a3-32195b8333e0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:49.876973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61194-2490-462e-826a-361a22f60961 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:49.877577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61194-2490-462e-826a-361a22f60961 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:49.877855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61194-2490-462e-826a-361a22f60961 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:49.878127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61194-2490-462e-826a-361a22f60961 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:49.879025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:49.879657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:49.879940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:49.880217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] PATH_INFO: `/v3/users/b0da6e03372642b8b2ee06780cd03714` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:49.881418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0da6e03372642b8b2ee06780cd03714)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:49.884591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:49.887339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de61194-2490-462e-826a-361a22f60961 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:49.888841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de61194-2490-462e-826a-361a22f60961 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:49.899444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1248/2488] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:46:49.937592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0da6e03372642b8b2ee06780cd03714'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:49.944452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0da6e03372642b8b2ee06780cd03714'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:49.945380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0da6e03372642b8b2ee06780cd03714'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:49.946002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0da6e03372642b8b2ee06780cd03714'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:49.954754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cb4bba0b-c5cf-4194-b0a5-f4f4d9b719fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0da6e03372642b8b2ee06780cd03714'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:49.974113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1241/2489] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:46:49 2022] DELETE /identity/v3/users/b0da6e03372642b8b2ee06780cd03714 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:46:50.138845 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61194-2490-462e-826a-361a22f60961 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.149970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61194-2490-462e-826a-361a22f60961 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.207477 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.208108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.208384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.208657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] PATH_INFO: `/v3/projects/ff0f334384454c7daba615e52ce21cd1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.209756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0f334384454c7daba615e52ce21cd1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.211702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.255944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0f334384454c7daba615e52ce21cd1', 'request_id': 'req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:50.261370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0f334384454c7daba615e52ce21cd1', 'request_id': 'req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:50.269489 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0f334384454c7daba615e52ce21cd1', 'request_id': 'req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:50.277433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:50.281848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] The token cache is being invalidate because project ff0f334384454c7daba615e52ce21cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:46:50.282342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:50.283460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d7d6f8d-8cf2-447b-910d-dd34873ba64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:46:50.285113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1249/2490] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:46:50 2022] DELETE /identity/v3/projects/ff0f334384454c7daba615e52ce21cd1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:46:50.290071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.290424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.290692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.310195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:50.311435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:50.376404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1242/2491] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:50.384104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.384393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.384743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.404723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:50.405946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:50.413936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1250/2492] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:46:50.418280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.428717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f9b26-0c90-479b-bf00-0b1bb1095e8b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.434229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.434959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.435305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.435615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.436280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:46:50.437195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.439129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.448132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:50.455535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1243/2493] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:46:50.456627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.466168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de7fea-e214-4866-91df-02ae908efd26 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.470952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.471553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.471860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.472136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.472779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.474758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.506451 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1251/2494] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:46:50 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:50.507730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.517988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cf047-dc86-41b4-93f7-42c60d4c0df4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.579807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.580405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.580685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.580960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.581588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.583557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.604209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1244/2495] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:46:50 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:46:50.608946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.619726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b18f10-22ae-4e0f-9f9b-66b2c5adfc54 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.624980 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.625578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.625855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.626124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.626783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.628392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.638748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1252/2496] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:50.643500 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.654384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bcf20-4080-401e-bd66-0a360d563e77 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.659056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.659591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.659876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.660149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.660796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.662391 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba6ab9-fa4a-439f-b6b4-da3d2c157d95 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.673482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1245/2497] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:46:50.677659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.688787 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0acb-259b-4c51-8d1c-c417046674c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.693798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.694463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.694751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.695029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] PATH_INFO: `/v3/projects/8e734660c5ff4ef8ac336feb74fae3aa/users/840e0c4327f24879baffe08b2dc1ff9d/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.710383 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e734660c5ff4ef8ac336feb74fae3aa, user_id=840e0c4327f24879baffe08b2dc1ff9d, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.713330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.726515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1253/2498] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:46:50 2022] PUT /identity/v3/projects/8e734660c5ff4ef8ac336feb74fae3aa/users/840e0c4327f24879baffe08b2dc1ff9d/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:46:50.737290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4601e-9e67-407c-bf76-9c86b9deee54 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.737810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4601e-9e67-407c-bf76-9c86b9deee54 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.737923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4601e-9e67-407c-bf76-9c86b9deee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.766447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3cc4601e-9e67-407c-bf76-9c86b9deee54 None None] MFA Rules not processed for user `840e0c4327f24879baffe08b2dc1ff9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:50.767721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3cc4601e-9e67-407c-bf76-9c86b9deee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:50.836318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1246/2499] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:50.848518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:50.850281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:50.909300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0dcec-1b4f-4d6d-a6e2-fca16082d9b3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:50.913199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:50.913802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:50.914076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:50.914342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:50.923531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:50.924937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:50.933726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1254/2500] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:57.013161 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c80bc-5d70-4e1a-8445-ed7c4c63d881 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:57.013741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c80bc-5d70-4e1a-8445-ed7c4c63d881 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:57.013741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c80bc-5d70-4e1a-8445-ed7c4c63d881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:57.033267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fb6c80bc-5d70-4e1a-8445-ed7c4c63d881 None None] MFA Rules not processed for user `88f34e4fc44e432d9e994ae5dcdbd1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:57.034524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fb6c80bc-5d70-4e1a-8445-ed7c4c63d881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:57.095031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1247/2501] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:46:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:00.929622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:07.086882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:11.946705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:11.948160 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:12.012648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404209-ae01-4a8a-aba7-566c767239fa service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:12.017816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:12.018454 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:12.018733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:12.019050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] PATH_INFO: `/v3/users/0f6470f8e28347fbb6259000136b140b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:12.020114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6470f8e28347fbb6259000136b140b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:12.022008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:12.068202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6470f8e28347fbb6259000136b140b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.074708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6470f8e28347fbb6259000136b140b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.075354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6470f8e28347fbb6259000136b140b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.075947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6470f8e28347fbb6259000136b140b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.083759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6470f8e28347fbb6259000136b140b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.101272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1255/2502] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:11 2022] DELETE /identity/v3/users/0f6470f8e28347fbb6259000136b140b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:12.105732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:12.106218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:12.106317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:12.125338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:12.126498 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:12.183691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1248/2503] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:12.193311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:12.194762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:12.255325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ed1c-189b-4bd1-b39c-8968082577c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:12.259791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:12.259791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:12.260036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:12.260115 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:12.272327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:12.273754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:12.284144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1256/2504] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:12.566765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:12.568202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:12.582144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68cbc5-f7d1-48e4-a54d-bc8c45351cf2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:12.587178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:12.588367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:12.588840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:12.589127 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] PATH_INFO: `/v3/projects/c34fc4f1109d4982882747f0f92477cb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:12.590352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34fc4f1109d4982882747f0f92477cb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:12.592205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:12.640288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34fc4f1109d4982882747f0f92477cb', 'request_id': 'req-1598060d-3e56-48c8-b391-0371c68e4c45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.647337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34fc4f1109d4982882747f0f92477cb', 'request_id': 'req-1598060d-3e56-48c8-b391-0371c68e4c45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.653938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34fc4f1109d4982882747f0f92477cb', 'request_id': 'req-1598060d-3e56-48c8-b391-0371c68e4c45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.664823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] The token cache is being invalidate because project c34fc4f1109d4982882747f0f92477cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:12.665334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.665992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:12.667589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1249/2505] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:12 2022] DELETE /identity/v3/projects/c34fc4f1109d4982882747f0f92477cb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:22.279151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:22.661389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:22.771464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.773039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.784507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eae761-1d52-4dc0-a886-8027c8866e0f service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.838717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.839278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.840219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.840591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.842644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.906707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.908139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.918619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1257/2506] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:32.913699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:43.204991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:43.206519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.218624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598060d-3e56-48c8-b391-0371c68e4c45 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:43.222802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:43.223403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:43.223717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:43.223995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:43.225668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.280740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:43.282715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.303687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:43.305110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9444e-4436-419b-8801-d10100c3995c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:43.316810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1250/2507] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:43.352236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525e761-c2bd-4f31-a6be-27ee318a09f9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:43.356989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:43.357620 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:43.357898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:43.358171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] PATH_INFO: `/v3/users/840e0c4327f24879baffe08b2dc1ff9d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:43.359409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=840e0c4327f24879baffe08b2dc1ff9d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:43.361385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:43.415504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840e0c4327f24879baffe08b2dc1ff9d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.422567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840e0c4327f24879baffe08b2dc1ff9d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.423272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840e0c4327f24879baffe08b2dc1ff9d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.423902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840e0c4327f24879baffe08b2dc1ff9d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.433218 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840e0c4327f24879baffe08b2dc1ff9d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.460145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1258/2508] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:43 2022] DELETE /identity/v3/users/840e0c4327f24879baffe08b2dc1ff9d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:47:43.464993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739433-9c13-459e-864c-c444fe40797a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:43.465292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739433-9c13-459e-864c-c444fe40797a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:43.465683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739433-9c13-459e-864c-c444fe40797a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:43.484807 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bf739433-9c13-459e-864c-c444fe40797a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:43.485986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf739433-9c13-459e-864c-c444fe40797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.542907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1251/2509] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:43.552624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:43.554055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.615322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a02bc4-e992-4c66-9d55-c62a49f10cbe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:43.619251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:43.619862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:43.620144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:43.620420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:43.629988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:43.631452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16b26b-01c9-40a2-b35d-7db736f6635c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:43.640863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1259/2510] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:43.912936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739433-9c13-459e-864c-c444fe40797a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:43.914298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf739433-9c13-459e-864c-c444fe40797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:43.928192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739433-9c13-459e-864c-c444fe40797a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:43.933762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:43.934672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:43.935121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:43.935531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] PATH_INFO: `/v3/projects/8e734660c5ff4ef8ac336feb74fae3aa` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:43.936860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e734660c5ff4ef8ac336feb74fae3aa)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:43.939378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:43.987098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e734660c5ff4ef8ac336feb74fae3aa', 'request_id': 'req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:43.994131 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e734660c5ff4ef8ac336feb74fae3aa', 'request_id': 'req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:44.000418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e734660c5ff4ef8ac336feb74fae3aa', 'request_id': 'req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:44.011138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] The token cache is being invalidate because project 8e734660c5ff4ef8ac336feb74fae3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:44.011450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:44.012113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0f296fb0-e9ec-4369-b7b1-0fc5b63a475d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:44.013632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1252/2511] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:43 2022] DELETE /identity/v3/projects/8e734660c5ff4ef8ac336feb74fae3aa => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:44.018546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.018830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.019122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.038655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:44.039826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.058365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:44.102570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1260/2512] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:44.111635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.111920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.112197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.131851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:44.133079 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.140987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1253/2513] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:47:44.145480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.155273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7665114-2aea-4bca-a9ae-a21291e26f1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.160287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.160881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.161163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.161437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.162092 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:47:44.162381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.164028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.175989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1261/2514] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:47:44.180260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.185014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:44.191187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5cdce-5477-44d0-af36-9e89f26dda20 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.196711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.197313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.197590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.197859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.198524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.200418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.227304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1254/2515] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:44 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:44.231497 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.242590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75884cb-fb3e-4471-86ed-a76118c89435 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.300015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.300670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.300895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.301172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.301810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.303796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.324952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1262/2516] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:44 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:44.329554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.339569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd1a9-3bcd-490f-b20a-098aa29786ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.344670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.345254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.345528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.345795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.346447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.348034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.357810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1255/2517] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:44.361851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.371899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512a4f-3e5b-4488-bca8-2b88704cbbf9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.376569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.377166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.377444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.377719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.378363 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.380299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185620-750a-4bba-b887-edb2e33a049f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.391338 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1263/2518] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:44.395534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.405531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9046e-2d4d-4cfd-9905-1a84beace8a6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.410787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.411429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.411710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.411984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] PATH_INFO: `/v3/projects/f3eec134d5864a3c82f27706eb8065fa/users/525cdc42a473486ba64b481cf2820d31/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.426487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3eec134d5864a3c82f27706eb8065fa, user_id=525cdc42a473486ba64b481cf2820d31, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.429144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.444428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1256/2519] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:44 2022] PUT /identity/v3/projects/f3eec134d5864a3c82f27706eb8065fa/users/525cdc42a473486ba64b481cf2820d31/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:47:44.449657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.449960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.450212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.470432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:44.471633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.514609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.516735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.532594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1264/2520] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:44.542876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:44.544676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.574064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b58b6-e1aa-426e-a6d8-af1432708fcd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.576446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:44.579532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.580169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.580450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.581417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.582908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:44.604768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113768-cfd4-420a-bbbf-970ef910af7c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:44.609593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:44.610199 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:44.610512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:44.610918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:44.620408 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.621854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cfaad-4342-4762-9c98-5fc4994fdabf service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.631742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1265/2521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:47:44.659855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:44.661312 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:44.672770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1257/2522] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:48.961361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:48.961920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:48.961920 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:48.994234 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] MFA Rules not processed for user `525cdc42a473486ba64b481cf2820d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:48.995731 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:49.083985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1266/2523] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:49.089097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.100374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12f780-8570-45fe-b52e-67112bf423fd service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.106809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.107563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.107972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.108330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.109048 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.111886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.141449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1258/2524] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:49 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:47:49.145360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.157672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dfaf4-c95f-489f-b471-24547c7c52f3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.273064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.273690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.273971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.274360 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.275043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.277000 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.302388 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1267/2525] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:49 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:49.306659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.317629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347951fa-6d3d-4695-b2a4-c960a5a83bd0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.322939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.323678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.324028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.324371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.325091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.326841 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.342007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1259/2526] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:49.346848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.357222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c980-5536-483a-8157-cba6ccda84b7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.362647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.363314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.363597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.363895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] PATH_INFO: `/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40/users/dcdf8ff919a24ed4824549e460778de4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.380198 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8574fd3e7f214ed18d7e378ac96fbd40, user_id=dcdf8ff919a24ed4824549e460778de4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.382925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.397821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1268/2527] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:49 2022] PUT /identity/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40/users/dcdf8ff919a24ed4824549e460778de4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:49.401664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.413610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261049-e9a8-4e5e-81ea-9118ad16e246 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.424676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:49.480740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.481354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.481638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.481919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.482618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.484822 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8eec5-c328-4e00-b5cf-a00095e0d00d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.495650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1260/2528] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:49.499946 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:49.510734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb490e-34fb-4b04-9b40-143f0fdb5280 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:49.515728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:49.516455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:49.516741 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:49.517068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] PATH_INFO: `/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40/users/dcdf8ff919a24ed4824549e460778de4/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:49.519049 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8574fd3e7f214ed18d7e378ac96fbd40, user_id=dcdf8ff919a24ed4824549e460778de4, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:49.521712 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:49.534513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1269/2529] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:49 2022] PUT /identity/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40/users/dcdf8ff919a24ed4824549e460778de4/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:52.605762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade247f-6c8d-4e5b-a18c-17d6da3de396 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:52.606221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade247f-6c8d-4e5b-a18c-17d6da3de396 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:52.606483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade247f-6c8d-4e5b-a18c-17d6da3de396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:52.646797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-aade247f-6c8d-4e5b-a18c-17d6da3de396 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:52.648388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aade247f-6c8d-4e5b-a18c-17d6da3de396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:52.745846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1261/2530] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:52.859601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:52.862040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:52.889172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:52.986752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc275b-d8db-424e-ae03-ec81ef3c4b0b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:52.992809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:52.993895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:52.994399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:52.995029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:53.007638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:53.010152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:53.023208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1270/2531] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:47:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:47:54.987176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e83b1-060a-493d-b35c-ab28a75bb9e5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:54.987700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e83b1-060a-493d-b35c-ab28a75bb9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:54.987788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e83b1-060a-493d-b35c-ab28a75bb9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:55.032524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f27e83b1-060a-493d-b35c-ab28a75bb9e5 None None] MFA Rules not processed for user `dcdf8ff919a24ed4824549e460778de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:55.036388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f27e83b1-060a-493d-b35c-ab28a75bb9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:55.184070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1262/2532] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:55.193576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:55.195298 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:55.298730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8acd2-a2cc-4e8a-a4f0-a014d7d7400f service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:55.303027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:55.303813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:55.304173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:55.304515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:55.380952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:55.382605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:55.395405 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1271/2533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:55.619629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:55.620447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:55.620725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:55.653719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:55.655148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:55.669279 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1263/2534] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:47:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:55.696601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:55.725661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8d165-c610-442a-a083-0c3161fb5475 service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:55.769869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:55.770633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:55.771684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:55.771684 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:55.828232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:55.829894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dedc98-ba5d-474a-8944-34ffd15e1005 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:55.843970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1272/2535] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:47:59.535113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:59.537768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:59.631152 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:59.735641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9675656-f98c-4bba-a162-dbedc8cdadeb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:59.741143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:59.741750 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:59.742026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:59.742299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:59.744056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:59.879319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:59.881971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:59.915243 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1264/2536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:05.838753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=80 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:07.571053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:07.571666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:07.571666 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:07.591098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] MFA Rules not processed for user `525cdc42a473486ba64b481cf2820d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:07.592276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:07.598981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1273/2537] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:07.742123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:07.802819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ffca6-bb64-4f24-a61c-7c173fe69e8f service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:07.806664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:07.807275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:07.807559 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:07.807836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:07.809597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:07.820813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:07.822482 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:07.831636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1265/2538] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:07.899906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:07.912071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf542f-5f2c-43d2-9b43-626922fdba9e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:07.915749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:07.916365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:07.916641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:07.916912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:07.919023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:07.929299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:07.931055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bef551-d25a-42af-abd2-92320ad21ed8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:07.939742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1274/2539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:09.300529 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.301905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:09.374617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9637c-9eb2-45c5-95b2-07c91277a6b3 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.379609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.380212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.380493 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.380766 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.382570 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:09.394257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.396032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440abf7-b6d8-47c5-9f14-461848c748e6 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.405998 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1266/2540] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:11.509297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.511312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.511757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.513876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1275/2541] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:48:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:48:11.532136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7019b-2171-4394-9f9e-88773d8d38af None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.532640 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7019b-2171-4394-9f9e-88773d8d38af None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.532757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7019b-2171-4394-9f9e-88773d8d38af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.541134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:11.557284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fff7019b-2171-4394-9f9e-88773d8d38af None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:11.558536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fff7019b-2171-4394-9f9e-88773d8d38af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.575146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1267/2542] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:11.583934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:11.586325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.606778 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c734-8655-4b36-a7d5-903acf558a94 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:11.612642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25cbe9fdae7b41ad9735c20260ffe22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:11.614036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.614337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.614817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.633211 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.638108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d7fdc-97d8-436c-be89-6d0161f57bdf service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.658028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1276/2543] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:11.776113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.776815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.777097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.779177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1268/2544] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:48:11.785857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa1f1-0e6d-40f6-ad90-fa69229d034b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.786149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa1f1-0e6d-40f6-ad90-fa69229d034b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.786440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa1f1-0e6d-40f6-ad90-fa69229d034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.808972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e3caa1f1-0e6d-40f6-ad90-fa69229d034b None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:11.810699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3caa1f1-0e6d-40f6-ad90-fa69229d034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.820271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1277/2545] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:11.833242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:11.835057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.849528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485d16f-da28-4f97-9119-cd3c24231710 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:11.854041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:11.854729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.855000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.855268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.863093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.864519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b22f-1cc8-4c48-998d-e4f03a3b41e0 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.872361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1269/2546] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:11.976950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.977237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.977514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.006019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:12.007469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.013779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8913-1796-4e13-9f74-8372559e8d22 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.014292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8913-1796-4e13-9f74-8372559e8d22 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.014717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8913-1796-4e13-9f74-8372559e8d22 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.015522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1278/2547] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:12.016572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1270/2548] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:48:12.026983 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.027520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.027959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.028837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.041692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cc0b-b877-4200-9244-45e39b4c18cd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.048504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.048676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:12.049116 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.049398 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.049677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.050057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.059435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.061247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.062690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1271/2549] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:12.070921 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1279/2550] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:48:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:48:12.075013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.086667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95f3cc-3ca6-4130-8d18-7be03af9017b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.090813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.091428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.091842 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.092436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.104343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.105795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca4143-2048-44e5-a374-12bf33699718 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.117946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1272/2551] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:22.067371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=27 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:22.113210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=17 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:24.880798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:24.882368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:24.899204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31f2b-ca1e-43e7-8d4a-e4d50bca12df service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:24.905878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:24.906561 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:24.906922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:24.907064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:24.908927 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:24.922460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:24.924029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9268f-5b49-4aff-8141-65319fe691c2 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:24.935110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1280/2552] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:28.584932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:28.585454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:28.585454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:28.604091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] MFA Rules not processed for user `d8581d46699041d5a4dcb8a43034863e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:28.605267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:28.612591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1273/2553] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:32.579064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:32.579621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:32.579621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:32.599750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:32.600950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:32.608259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1281/2554] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:32.621397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:32.636955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d641d6-22b8-480d-b8ea-7a68a4872d83 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:32.642537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:32.643136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:32.643414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:32.643685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:32.652008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:32.653588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:32.662348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1274/2555] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:48:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:48:42.594495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=9 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:42.659087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=9 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:50.405589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.419170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa6bf8-093b-4198-94e2-c284b4dad779 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.424902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.425605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.425957 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.426305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.431553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:50.445143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.446751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5da5f2-6134-4b86-a49c-0dedaa73ee3a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.457998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1282/2556] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:58.895884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:58.909543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889cf2a-dd0e-4537-a9ca-9fb2f205b21a service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:58.915080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:58.915675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:58.915952 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:58.916220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:58.917985 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:58.930815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:58.932405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:58.944262 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1275/2557] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:49:00.453938 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:01.495357 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.495967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.495967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.516274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:01.517495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.524695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1283/2558] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:01.549235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.561418 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435908e-a68f-4e0c-97fc-eafc8116a49e service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.566501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.567108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.567389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.567666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.576729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.578329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.590670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1276/2559] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:49:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:49:04.289672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:04.298954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e21c2-0baa-42d6-8724-bbdca2554e0e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:04.302565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:04.303156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:04.303431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:04.303704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:04.313361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:04.314911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209804ba-5e84-4849-82ef-e34006f7f544 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:04.324082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1284/2560] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:11.583489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=12 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:14.320034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=9 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:27.676765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:27.686458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b555e-6149-495d-b2de-484eabfb9770 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:27.689988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:27.690610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:27.690892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:27.691164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:27.692878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:27.702071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:27.703515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cae3ca-1ab2-429c-a1ce-4fbf2d6d29e2 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:27.711316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1277/2561] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:28.841885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd0aa-6373-4f12-be68-f2094172f023 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:28.842555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd0aa-6373-4f12-be68-f2094172f023 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:28.842654 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd0aa-6373-4f12-be68-f2094172f023 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:28.843941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1285/2562] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:49:28.850380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9750-2d99-4050-9c66-21d9177137ef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:28.851073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9750-2d99-4050-9c66-21d9177137ef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:28.851073 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9750-2d99-4050-9c66-21d9177137ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:28.871232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd3b9750-2d99-4050-9c66-21d9177137ef None None] MFA Rules not processed for user `1393d3a8c9174d08b2c6d63cf4e25b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:28.873108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd3b9750-2d99-4050-9c66-21d9177137ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:28.879989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1278/2563] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:28.894242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd0aa-6373-4f12-be68-f2094172f023 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:28.906201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd0aa-6373-4f12-be68-f2094172f023 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:28.909821 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:28.910442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:28.910733 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:28.911029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:28.919050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:28.920474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c18012-f0d8-4adc-a0b6-288581bba38a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:28.928575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1286/2564] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:29.021745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa9b1-31cc-434a-ad74-effb32251085 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:29.022119 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa9b1-31cc-434a-ad74-effb32251085 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:29.022484 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa9b1-31cc-434a-ad74-effb32251085 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:29.023839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1279/2565] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 15:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:49:29.030215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d451205-3808-416e-a524-b072bc4f786b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:29.030568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d451205-3808-416e-a524-b072bc4f786b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:29.030833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d451205-3808-416e-a524-b072bc4f786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:29.050941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5d451205-3808-416e-a524-b072bc4f786b None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:29.052082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5d451205-3808-416e-a524-b072bc4f786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:29.061882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1287/2566] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 15:49:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:29.073978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daa9b1-31cc-434a-ad74-effb32251085 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:29.089930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daa9b1-31cc-434a-ad74-effb32251085 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:29.094240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:29.094895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:29.095168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:29.095442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:29.106978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:29.108419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:29.119600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1280/2567] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:39.045789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=9 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:39.114681 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=15 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:07.231863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:07.232882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:07.233259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:07.254350 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] MFA Rules not processed for user `88f34e4fc44e432d9e994ae5dcdbd1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:07.255735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:07.322704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1288/2568] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:10.871064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:10.885858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92b66c-6e6a-442e-92e5-8623e4393cdc service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:10.892936 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:10.893540 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:10.893819 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:10.894091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:10.896053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:10.958537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:10.960132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4f57-ac59-4b97-afd5-cb25e915179e service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:10.969408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1281/2569] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:50:17.315918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:20.443054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:20.452834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e249c-298c-4daf-b501-2c148596d1d6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:20.456500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:20.457146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:20.457473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:20.457728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:20.459515 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:20.527050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:20.528639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:20.538917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1289/2570] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:20.965065 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=28 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:22.269815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:22.270363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:22.270363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:22.300178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] MFA Rules not processed for user `9ecdfa3c79f84cc29061dd5ebfa60101`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:22.301388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:22.310462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1282/2571] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:24.389455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:24.408828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63dcd-9e44-46a0-9244-111a6a8ba8fb service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:24.427255 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:24.427857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:24.428331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:24.429096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:24.432555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:24.451245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:24.453048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:24.486984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1290/2572] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:24.941034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:24.942785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:24.973632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9990d-dc63-4b5d-87d7-56363192956f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:24.978903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:24.979556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:24.979830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:24.980129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] PATH_INFO: `/v3/users/d8581d46699041d5a4dcb8a43034863e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:24.981872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8581d46699041d5a4dcb8a43034863e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:24.983942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.039169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8581d46699041d5a4dcb8a43034863e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.047222 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8581d46699041d5a4dcb8a43034863e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.048193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8581d46699041d5a4dcb8a43034863e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.048799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8581d46699041d5a4dcb8a43034863e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.057154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-eee14b21-4735-46fc-bc53-3cb1524ebd39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8581d46699041d5a4dcb8a43034863e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.077115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1283/2573] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:24 2022] DELETE /identity/v3/users/d8581d46699041d5a4dcb8a43034863e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:25.286178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.301960 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52d36c-a8a5-4a06-a064-134bfc6b49e0 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.384393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.385085 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.385367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.385646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] PATH_INFO: `/v3/projects/22d9fd2a9bec45938dd18540b1334a3b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.387644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d9fd2a9bec45938dd18540b1334a3b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.390227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.461609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d9fd2a9bec45938dd18540b1334a3b', 'request_id': 'req-1f6821a2-dff1-4d76-9658-5032d61ed38f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.470114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d9fd2a9bec45938dd18540b1334a3b', 'request_id': 'req-1f6821a2-dff1-4d76-9658-5032d61ed38f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.481610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d9fd2a9bec45938dd18540b1334a3b', 'request_id': 'req-1f6821a2-dff1-4d76-9658-5032d61ed38f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.500763 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] The token cache is being invalidate because project 22d9fd2a9bec45938dd18540b1334a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:25.501095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.501766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f6821a2-dff1-4d76-9658-5032d61ed38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:25.503699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1291/2574] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:25 2022] DELETE /identity/v3/projects/22d9fd2a9bec45938dd18540b1334a3b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:25.509966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.510741 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.512547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.540652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:25.542227 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:25.620505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1284/2575] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:25.628967 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.629252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.629706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.658653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:25.660532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:25.672246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1292/2576] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:25.676255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.687328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8214d3-2c77-4a67-9981-e2c2fcca9570 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.692350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.693832 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.694609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.695122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.696132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:25.696660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.700456 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.717301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1285/2577] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:25.721911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.727887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:25.733434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d949cf-bfbc-4c5b-a987-ee6372443b23 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.740306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.741226 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.741519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.742903 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.743248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.745419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.779367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1293/2578] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:25 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:25.785586 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.811124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb3cc0-895f-4c07-a077-9f7a642917c7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.880663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:25.895254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.895887 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.896176 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.896504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.897747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.900707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.930610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1286/2579] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:25 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:25.932776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.943554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced279bf-3de6-4cd7-ae69-6d934ef019fe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.949419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.949985 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.950270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.950624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.951284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.952930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:25.968593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1294/2580] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:25.972483 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:25.983485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c01d2-02fb-49cd-a138-88ee78683fc0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:25.988250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:25.988855 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:25.989136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:25.989411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:25.990052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:25.991699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757496e-9bf7-4dd9-a78a-b5776533e64d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.002828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1287/2581] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:26.007580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.017988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9725-f840-45ef-a77d-9451c3a9d9cd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.022992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.023828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.024174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.024510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] PATH_INFO: `/v3/projects/a527e0d26b194f35804dcfba406e1e8d/users/617aec78f9744c7da3bd3c36b3f1ba16/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.042377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a527e0d26b194f35804dcfba406e1e8d, user_id=617aec78f9744c7da3bd3c36b3f1ba16, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.045972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.062225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1295/2582] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:26 2022] PUT /identity/v3/projects/a527e0d26b194f35804dcfba406e1e8d/users/617aec78f9744c7da3bd3c36b3f1ba16/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:26.073270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.073568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.073845 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.103756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] MFA Rules not processed for user `617aec78f9744c7da3bd3c36b3f1ba16`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:26.106294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.184511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1288/2583] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:50:26.189169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.260492 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcd7fb-57b6-48db-9575-74437b9a554d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.267705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.268465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.268748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.269026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.269664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.271625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.305216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1296/2584] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:26 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:26.309963 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.320949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ace15e-6da9-47dd-ba1c-ff07c51c385a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.379229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.379846 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.380126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.380401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.381260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.383770 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.406554 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1289/2585] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:26 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:26.410483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.421767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02ad5-46a4-42cd-80f9-23888f3d9741 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.427275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.427992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.428344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.428933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.429655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.431310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.443196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1297/2586] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:50:26.447523 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.458197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a576d7-9bbe-4d71-9c6e-3781b66e36a2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.464299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.465334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.465334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.465658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] PATH_INFO: `/v3/projects/efda11391b6e42749acdc906a41e01f4/users/eb24e8eca1a442a2b06c386bbd094e45/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.494627 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efda11391b6e42749acdc906a41e01f4, user_id=eb24e8eca1a442a2b06c386bbd094e45, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.499530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.519060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1290/2587] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:26 2022] PUT /identity/v3/projects/efda11391b6e42749acdc906a41e01f4/users/eb24e8eca1a442a2b06c386bbd094e45/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:26.520040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.530367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b5a19-d296-4310-b2f1-e39c76062256 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.596032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.596516 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.596861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.597201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.598022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.599611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ce3dd-eb9d-44db-b1ce-71a9effd8e0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.616796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1298/2588] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:26.618685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.632501 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ce94-9021-4466-adc8-f75530c97587 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.637518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.638244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.638615 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.638973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] PATH_INFO: `/v3/projects/efda11391b6e42749acdc906a41e01f4/users/eb24e8eca1a442a2b06c386bbd094e45/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.641217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efda11391b6e42749acdc906a41e01f4, user_id=eb24e8eca1a442a2b06c386bbd094e45, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.643986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.656945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1291/2589] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:26 2022] PUT /identity/v3/projects/efda11391b6e42749acdc906a41e01f4/users/eb24e8eca1a442a2b06c386bbd094e45/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:26.669345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.669724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.670616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.703768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] MFA Rules not processed for user `eb24e8eca1a442a2b06c386bbd094e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:26.704894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.710692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:26.789428 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1299/2590] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:26.801003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.802751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.819296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:26.867292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff2411-d885-4687-af22-c03398d2c7c2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.871470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.872088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.872365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.872646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.882997 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.884328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a3057-47da-4546-b728-8d65da9f184e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.895215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1292/2591] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:27.074036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:27.136052 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9001f65-c9ec-49fc-bf35-d99abc257603 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:27.140896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:27.141529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:27.141808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:27.142082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] PATH_INFO: `/v3/users/617aec78f9744c7da3bd3c36b3f1ba16` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:27.143262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617aec78f9744c7da3bd3c36b3f1ba16)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:27.145238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:27.193002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617aec78f9744c7da3bd3c36b3f1ba16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.200060 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617aec78f9744c7da3bd3c36b3f1ba16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.200735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617aec78f9744c7da3bd3c36b3f1ba16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.201335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617aec78f9744c7da3bd3c36b3f1ba16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.208467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617aec78f9744c7da3bd3c36b3f1ba16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.226310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1300/2592] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:27 2022] DELETE /identity/v3/users/617aec78f9744c7da3bd3c36b3f1ba16 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:27.231351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:27.231888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:27.231984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:27.252040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:27.253245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:27.315409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1293/2593] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:27.327022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:27.328565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:27.359768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:27.405719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99475c89-506e-45fb-9db5-4fd720b5da65 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:27.409522 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:27.410154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:27.410465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:27.410736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:27.421090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:27.422555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:27.432708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1301/2594] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:27.751419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:27.752872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:27.766028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7a613-7d9a-41bc-9124-885210a74de6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:27.775758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:27.776399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:27.776683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:27.776995 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] PATH_INFO: `/v3/projects/a527e0d26b194f35804dcfba406e1e8d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:27.778652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a527e0d26b194f35804dcfba406e1e8d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:27.780563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:27.834711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a527e0d26b194f35804dcfba406e1e8d', 'request_id': 'req-7516021c-0f71-496d-97ed-c602aac92b73'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.845230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a527e0d26b194f35804dcfba406e1e8d', 'request_id': 'req-7516021c-0f71-496d-97ed-c602aac92b73'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.852313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a527e0d26b194f35804dcfba406e1e8d', 'request_id': 'req-7516021c-0f71-496d-97ed-c602aac92b73'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.864155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] The token cache is being invalidate because project a527e0d26b194f35804dcfba406e1e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:27.864530 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.865281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:27.866892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1294/2595] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:27 2022] DELETE /identity/v3/projects/a527e0d26b194f35804dcfba406e1e8d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:27.871040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:27.892530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2ad8a-eb93-4282-b32b-092809b70533 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:27.964533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:27.965711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:27.965992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:27.966315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] PATH_INFO: `/v3/users/eb24e8eca1a442a2b06c386bbd094e45` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:27.967490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb24e8eca1a442a2b06c386bbd094e45)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:27.969434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:28.029612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb24e8eca1a442a2b06c386bbd094e45'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.037303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb24e8eca1a442a2b06c386bbd094e45'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.038037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb24e8eca1a442a2b06c386bbd094e45'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.038699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb24e8eca1a442a2b06c386bbd094e45'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.046871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e008e49b-af79-43a2-93cb-357cf423d438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb24e8eca1a442a2b06c386bbd094e45'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.064840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1302/2596] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:27 2022] DELETE /identity/v3/users/eb24e8eca1a442a2b06c386bbd094e45 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:28.486782 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:28.488982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:28.505531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516021c-0f71-496d-97ed-c602aac92b73 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:28.550824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:28.574625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:28.575260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.575544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.575818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] PATH_INFO: `/v3/projects/efda11391b6e42749acdc906a41e01f4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.577341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] RBAC: Authorizing `identity:delete_project(project_id=efda11391b6e42749acdc906a41e01f4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:28.580039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:28.626646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efda11391b6e42749acdc906a41e01f4', 'request_id': 'req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.633940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efda11391b6e42749acdc906a41e01f4', 'request_id': 'req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.641463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efda11391b6e42749acdc906a41e01f4', 'request_id': 'req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.652812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] The token cache is being invalidate because project efda11391b6e42749acdc906a41e01f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:28.653109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.653760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95b770bf-4e4c-42bd-9e3e-69fd7c7c802a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:28.655289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1295/2597] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:28 2022] DELETE /identity/v3/projects/efda11391b6e42749acdc906a41e01f4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:28.660337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.660793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.660916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.681276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:28.682491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:28.723616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:28.744253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1303/2598] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:28.753330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.753651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.754084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.774387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:28.775632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:28.784747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1296/2599] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:28.788981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:28.800990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f7d44-2e30-43d6-a84f-af1728a5d2c0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:28.806216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:28.806858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.807145 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.807421 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.808303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:28.808605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:28.810224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:28.822965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1304/2600] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:28.827583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:28.838499 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b8c36-9722-4810-ac5e-e8d4163d5651 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:28.845713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:28.846360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.846689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.846969 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.847729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:28.849709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:28.879809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1297/2601] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:28 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:28.887222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:28.896230 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed555-b3c2-463f-8afb-db242a194b8f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:28.956229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:28.956833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:28.957291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:28.957567 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:28.958202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:28.960134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:28.981479 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1305/2602] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:28 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:28.985660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:28.997779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfa48b-20ee-4aac-b5e9-d2c75cb8a5b7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.007181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.008226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.008779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.009286 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.010507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.017356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.031538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1298/2603] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:29.035374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.051766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39970cb-4159-4b38-8557-7d029d2c83c6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.062251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.062619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.062973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.063337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.064037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.065829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d049468-fa74-4453-b945-084a4c57e444 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.081774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1306/2604] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:29.084341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.095281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83831a-96a7-4e36-bff5-2cac80e12667 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.101006 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.101742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.101940 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.102288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] PATH_INFO: `/v3/projects/07ed068a37df4fd490a48903f1186f4d/users/206d82a8f8e54b02a4c56099f4f270f2/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.117895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ed068a37df4fd490a48903f1186f4d, user_id=206d82a8f8e54b02a4c56099f4f270f2, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.121173 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.135186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1299/2605] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:29 2022] PUT /identity/v3/projects/07ed068a37df4fd490a48903f1186f4d/users/206d82a8f8e54b02a4c56099f4f270f2/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:50:29.146260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.146533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.146812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.175766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] MFA Rules not processed for user `206d82a8f8e54b02a4c56099f4f270f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:29.177091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:29.252943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1307/2606] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:29.257599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.332617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c170798-6d55-4763-9d00-9aa9cd982bc5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.338359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.339004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.339289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.339612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.340261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.342159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.353656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:29.375450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1300/2607] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:29 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:29.376657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.387440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a0d38-381d-475e-9ade-0f1f5931166a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.426349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:29.447892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.448514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.449369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.449640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.450278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.452278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.476773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1308/2608] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:29 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:29.477927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.488563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be553-65bb-4ddc-8d2b-14c023e0f10a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.493666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.494268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.494580 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.494916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.495547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.497221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.508618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1301/2609] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:29.513465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.523950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095265aa-f75b-44c1-9fcb-5abf96c559b7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.529159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.529814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.530095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.530371 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] PATH_INFO: `/v3/projects/4482dfb5c36649c48fbe050f50b8b59f/users/a277b67412714232b1f4d3c293202959/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.545797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4482dfb5c36649c48fbe050f50b8b59f, user_id=a277b67412714232b1f4d3c293202959, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.548530 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.563633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1309/2610] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:29 2022] PUT /identity/v3/projects/4482dfb5c36649c48fbe050f50b8b59f/users/a277b67412714232b1f4d3c293202959/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:50:29.567892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.578924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7e8d-80b8-4981-8185-8d4834e78840 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.639184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.639908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.640194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.640467 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.641140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.642857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c829f8-63d8-44a1-89ff-ebb7a10e0ba1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.652994 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1302/2611] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:29.658023 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.670668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9340-766e-460d-a51f-5d0844b9b995 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.675984 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.676751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.677036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.677354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] PATH_INFO: `/v3/projects/4482dfb5c36649c48fbe050f50b8b59f/users/a277b67412714232b1f4d3c293202959/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.679841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4482dfb5c36649c48fbe050f50b8b59f, user_id=a277b67412714232b1f4d3c293202959, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.682930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.694991 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1310/2612] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:29 2022] PUT /identity/v3/projects/4482dfb5c36649c48fbe050f50b8b59f/users/a277b67412714232b1f4d3c293202959/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:29.707039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.707392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.707654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.737004 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] MFA Rules not processed for user `a277b67412714232b1f4d3c293202959`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:29.738233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:29.810304 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1303/2613] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:29.821869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:29.823621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:29.883750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f1e42-4ea0-44f1-a3f8-5be81ca4c709 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:29.887963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:29.888568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:29.888844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:29.889117 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:29.898481 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:29.900026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0ed7-e659-40cd-86d5-ee61b50bb786 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:29.909779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1311/2614] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:30.380028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:30.466856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da317a0a-703f-461b-a20d-226c24a7230c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:30.472025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:30.472278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:30.472518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:30.472787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] PATH_INFO: `/v3/users/206d82a8f8e54b02a4c56099f4f270f2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:30.474258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206d82a8f8e54b02a4c56099f4f270f2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:30.476240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:30.480825 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:30.533686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206d82a8f8e54b02a4c56099f4f270f2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:30.541395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206d82a8f8e54b02a4c56099f4f270f2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:30.542165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206d82a8f8e54b02a4c56099f4f270f2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:30.543876 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206d82a8f8e54b02a4c56099f4f270f2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:30.560054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206d82a8f8e54b02a4c56099f4f270f2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:30.579212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1304/2615] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:30 2022] DELETE /identity/v3/users/206d82a8f8e54b02a4c56099f4f270f2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:30.583713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:30.584001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:30.584275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:30.604940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:30.607417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:30.668433 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1312/2616] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:30.680437 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:30.681899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:30.741491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0ec9-6132-4bc3-91b1-59369e554e90 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:30.745095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:30.745727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:30.746000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:30.746269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:30.755360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:30.756780 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:30.767553 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1305/2617] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:31.106734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:31.109015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:31.124655 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bc1de-f7d5-49fb-ad8e-4a0560e976d6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:31.136828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:31.137699 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:31.138051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:31.138399 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] PATH_INFO: `/v3/projects/07ed068a37df4fd490a48903f1186f4d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:31.139689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ed068a37df4fd490a48903f1186f4d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:31.142024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:31.185892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:31.202867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ed068a37df4fd490a48903f1186f4d', 'request_id': 'req-f4ef542c-78d8-4017-9df7-9343a611d705'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.210995 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ed068a37df4fd490a48903f1186f4d', 'request_id': 'req-f4ef542c-78d8-4017-9df7-9343a611d705'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.220110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ed068a37df4fd490a48903f1186f4d', 'request_id': 'req-f4ef542c-78d8-4017-9df7-9343a611d705'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.233327 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] The token cache is being invalidate because project 07ed068a37df4fd490a48903f1186f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:31.233676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.234281 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.238352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1313/2618] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:31 2022] DELETE /identity/v3/projects/07ed068a37df4fd490a48903f1186f4d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:31.240200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:31.252903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e4fff-ea38-431c-98b7-1aecbc19c351 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:31.335250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:31.336166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:31.336541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:31.336908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] PATH_INFO: `/v3/users/a277b67412714232b1f4d3c293202959` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:31.338162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a277b67412714232b1f4d3c293202959)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:31.340812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:31.372894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:31.397886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a277b67412714232b1f4d3c293202959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.406930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a277b67412714232b1f4d3c293202959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.408401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a277b67412714232b1f4d3c293202959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.408623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a277b67412714232b1f4d3c293202959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.416342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-90f0247e-9de0-40ad-aa90-34cec9b0394c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a277b67412714232b1f4d3c293202959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.440674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1306/2619] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:31 2022] DELETE /identity/v3/users/a277b67412714232b1f4d3c293202959 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:50:31.774007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:31.784709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:31.804572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef542c-78d8-4017-9df7-9343a611d705 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:31.872784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:31.873469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:31.873745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:31.874099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] PATH_INFO: `/v3/projects/4482dfb5c36649c48fbe050f50b8b59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:31.875222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4482dfb5c36649c48fbe050f50b8b59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:31.877611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:31.931247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4482dfb5c36649c48fbe050f50b8b59f', 'request_id': 'req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.936850 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4482dfb5c36649c48fbe050f50b8b59f', 'request_id': 'req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.944557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4482dfb5c36649c48fbe050f50b8b59f', 'request_id': 'req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.956050 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] The token cache is being invalidate because project 4482dfb5c36649c48fbe050f50b8b59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:31.956337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.956974 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05bb9dfa-2cd2-474f-83fe-87adaf665e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:31.958642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1314/2620] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:31 2022] DELETE /identity/v3/projects/4482dfb5c36649c48fbe050f50b8b59f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:31.963694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:31.964188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:31.964853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:31.988274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:31.989591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:32.059046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1307/2621] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:32.067557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.068107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.068605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.092780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:32.094003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:32.102108 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1315/2622] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:50:32.107254 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.118028 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c59e0-a97e-4e47-a5a6-77ad1153d3c5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.123124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.123676 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.124036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.124277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.124880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:32.125188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.126847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.139900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1308/2623] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:32.143926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.154602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3852-2848-4229-bd1b-14caaa34571f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.159409 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.160047 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.160336 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.160609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.161264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.163236 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.191630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1316/2624] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:32.195878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.206899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469bbd3-ead1-4bbd-95c2-1186b12e6cc3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.267376 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.268162 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.268449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.268725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.269383 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.271450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.302093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1309/2625] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:32.306788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.317988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e977c5-96e7-419b-8e6f-f59593f579ad admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.323014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.323692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.325089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.325089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.325089 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.326506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.339728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1317/2626] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:32.340942 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.351736 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e97f-c884-4937-b44b-574d4f2995a7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.357494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.358359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.358674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.358951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.359725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.361349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22771da3-e18d-4362-bf9d-15b486beb786 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.374590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1310/2627] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:32.378705 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.390063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1ba3-19d5-452b-913a-d22fcae0cc5a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.395177 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.395838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.396121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.396445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] PATH_INFO: `/v3/projects/30f7fbc4bab748ea9362eb3324358a4a/users/42aa29c1eb5a438eaf07c99d4cc27a1b/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.402035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:32.414769 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f7fbc4bab748ea9362eb3324358a4a, user_id=42aa29c1eb5a438eaf07c99d4cc27a1b, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.418656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.441679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1318/2628] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:32 2022] PUT /identity/v3/projects/30f7fbc4bab748ea9362eb3324358a4a/users/42aa29c1eb5a438eaf07c99d4cc27a1b/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:50:32.452167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.452369 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.452619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.478968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:32.491085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] MFA Rules not processed for user `42aa29c1eb5a438eaf07c99d4cc27a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:32.492327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:32.557609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1311/2629] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:32.562861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.628795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d79a0-2673-4acf-917b-ad90cfaf132e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.633361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.633968 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.634249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.634557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.635203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.637394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.665037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1319/2630] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:32.669746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.683195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb448-eea6-4a71-bdec-999d4b541a84 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.743671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.748907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.749763 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.750245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.751021 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.753428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.786388 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1312/2631] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:32 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:32.789715 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.801148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65bcea-9773-4748-8e6f-08353c4f6cae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.806876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.807528 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.807746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.808107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.808742 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.810503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.821475 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1320/2632] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:32.825783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.836521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e4b4e-e1ff-4d1a-98f7-7cba18b6c174 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.856217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.856908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.857187 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.857491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] PATH_INFO: `/v3/projects/d588d6a5e07a4496850cb9ae11f82850/users/f9dba7ed4c154b4c9d557042a15e1ad1/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.875939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d588d6a5e07a4496850cb9ae11f82850, user_id=f9dba7ed4c154b4c9d557042a15e1ad1, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.879011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.894135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1313/2633] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:32 2022] PUT /identity/v3/projects/d588d6a5e07a4496850cb9ae11f82850/users/f9dba7ed4c154b4c9d557042a15e1ad1/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:32.899163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:32.911494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485299-cae8-420c-a2fa-117a86fe985a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:32.976242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:32.976856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:32.977138 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:32.977414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:32.978058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:32.979687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ca732-6b49-4163-8c19-615f284c5e6c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:32.989891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1321/2634] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:32.994113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:33.008716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25126dea-5a75-4cab-bd78-5a86b9a84c2a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:33.014404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:33.015100 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:33.015384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:33.015662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] PATH_INFO: `/v3/projects/d588d6a5e07a4496850cb9ae11f82850/users/f9dba7ed4c154b4c9d557042a15e1ad1/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:33.017607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d588d6a5e07a4496850cb9ae11f82850, user_id=f9dba7ed4c154b4c9d557042a15e1ad1, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:33.020289 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:33.031571 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1314/2635] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:32 2022] PUT /identity/v3/projects/d588d6a5e07a4496850cb9ae11f82850/users/f9dba7ed4c154b4c9d557042a15e1ad1/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:33.042941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:33.043318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:33.043597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:33.070953 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] MFA Rules not processed for user `f9dba7ed4c154b4c9d557042a15e1ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:33.072147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:33.117331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:33.143482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1322/2636] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:33.155848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:33.157535 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:33.214258 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260801-286f-48d0-9b9c-6667e4aac56a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:33.218556 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:33.219156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:33.219438 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:33.219707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:33.225477 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:33.229744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:33.231244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:33.241320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1315/2637] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:33.288812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:33.290220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:33.302536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bde1c4-aecc-4830-adc7-85a68524500a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:33.306708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:33.307313 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:33.307593 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:33.307878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:33.365148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:33.366604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c995-7cd5-46dd-acc9-228f1e8b1fd9 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:33.375765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1323/2638] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:36.728760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:36.794833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630baa-03fc-4d4f-ac82-dc2dfa422365 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:36.799507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:36.800142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:36.800423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:36.800696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] PATH_INFO: `/v3/users/42aa29c1eb5a438eaf07c99d4cc27a1b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:36.801821 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42aa29c1eb5a438eaf07c99d4cc27a1b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:36.804042 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:36.854123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42aa29c1eb5a438eaf07c99d4cc27a1b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:36.861426 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42aa29c1eb5a438eaf07c99d4cc27a1b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:36.862232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42aa29c1eb5a438eaf07c99d4cc27a1b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:36.862968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42aa29c1eb5a438eaf07c99d4cc27a1b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:36.871469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42aa29c1eb5a438eaf07c99d4cc27a1b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:36.893351 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1316/2639] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:36 2022] DELETE /identity/v3/users/42aa29c1eb5a438eaf07c99d4cc27a1b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:36.897972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:36.898533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:36.898633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:36.919195 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:36.920443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:36.985698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1324/2640] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:36.996172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:36.997724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:37.062174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37380157-68b4-4997-8d4f-a20b89c14115 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:37.065954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:37.066606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:37.066885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:37.067156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:37.076827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:37.078275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:37.088332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1317/2641] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:37.374090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:37.375676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:37.392011 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913a7c-9860-413e-ba8d-cefa9f345fc7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:37.397855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:37.398766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:37.399144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:37.399564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] PATH_INFO: `/v3/projects/30f7fbc4bab748ea9362eb3324358a4a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:37.401001 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f7fbc4bab748ea9362eb3324358a4a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:37.403855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:37.456438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f7fbc4bab748ea9362eb3324358a4a', 'request_id': 'req-ebf38669-85bc-490a-884f-882f0222d6f1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.462604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f7fbc4bab748ea9362eb3324358a4a', 'request_id': 'req-ebf38669-85bc-490a-884f-882f0222d6f1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.471245 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f7fbc4bab748ea9362eb3324358a4a', 'request_id': 'req-ebf38669-85bc-490a-884f-882f0222d6f1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.483855 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] The token cache is being invalidate because project 30f7fbc4bab748ea9362eb3324358a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:37.484144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.484795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.486273 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1325/2642] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:37 2022] DELETE /identity/v3/projects/30f7fbc4bab748ea9362eb3324358a4a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:37.490405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:37.499968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0075-dc93-4fb9-9198-2400efdcf639 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:37.513337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:37.561788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:37.562436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:37.562732 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:37.563003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] PATH_INFO: `/v3/users/f9dba7ed4c154b4c9d557042a15e1ad1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:37.564121 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9dba7ed4c154b4c9d557042a15e1ad1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:37.566025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:37.616966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9dba7ed4c154b4c9d557042a15e1ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.624115 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9dba7ed4c154b4c9d557042a15e1ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.624790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9dba7ed4c154b4c9d557042a15e1ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.625401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9dba7ed4c154b4c9d557042a15e1ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.633181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-31e9b157-0d2e-4232-8db5-75a69eb8f775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9dba7ed4c154b4c9d557042a15e1ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:37.654003 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1318/2643] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:37 2022] DELETE /identity/v3/users/f9dba7ed4c154b4c9d557042a15e1ad1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:37.947194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:37.950951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:37.960901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:37.971147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38669-85bc-490a-884f-882f0222d6f1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.040119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.041355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.041355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.041355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] PATH_INFO: `/v3/projects/d588d6a5e07a4496850cb9ae11f82850` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.042956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d588d6a5e07a4496850cb9ae11f82850)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.045860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.112058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd588d6a5e07a4496850cb9ae11f82850', 'request_id': 'req-bce08a7f-c581-4cc0-9944-6493a4954b1e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:38.119162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd588d6a5e07a4496850cb9ae11f82850', 'request_id': 'req-bce08a7f-c581-4cc0-9944-6493a4954b1e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:38.127575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd588d6a5e07a4496850cb9ae11f82850', 'request_id': 'req-bce08a7f-c581-4cc0-9944-6493a4954b1e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:38.143838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] The token cache is being invalidate because project d588d6a5e07a4496850cb9ae11f82850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:38.144139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:38.144858 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bce08a7f-c581-4cc0-9944-6493a4954b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:38.146395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1326/2644] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:37 2022] DELETE /identity/v3/projects/d588d6a5e07a4496850cb9ae11f82850 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:38.152056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.152638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.152738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.174962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:38.176323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:38.244978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1319/2645] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:38.257127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.257413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.258106 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.283249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:38.284460 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:38.293688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1327/2646] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:38.298062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.308112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a9d3d-0d4e-4d53-a7ca-ce89ea4ad4ed None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.313455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.314242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.314402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.314698 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.315345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:38.315662 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.317256 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.330130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1320/2647] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:38.334501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.347292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451a5b4-adf6-4ce1-8d90-6ca7e332e210 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.353163 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.353753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.354035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.354436 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.355616 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.357681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.388254 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1328/2648] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:38.392454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.403954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efdb3-6558-4feb-bd31-457efc2ea795 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.425785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:38.478747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.479361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.480172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.480444 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.481076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.483113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.508037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1321/2649] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:38.511962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.523730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c200c4-6f50-4ed3-8deb-78700a921d04 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.541586 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.542901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.543318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.543689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.544462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.546167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.557419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1329/2650] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:38.562302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.573706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af16c4-0c0a-4b04-aae3-6c657ffa3e12 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.578381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.579046 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.579451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.579728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.580370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.581953 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd442bd-5a19-4f46-b520-089732268879 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.593225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1322/2651] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:50:38.597339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.608533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65b845-48d1-49f7-b9f6-324d70d3b51b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.614579 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.615716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.616002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.616278 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] PATH_INFO: `/v3/projects/8874969c142e4da89a95e46d23e55526/users/22f645a9d4f8477eac627c5202e91e10/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.631797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8874969c142e4da89a95e46d23e55526, user_id=22f645a9d4f8477eac627c5202e91e10, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.634482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.648314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1330/2652] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:38 2022] PUT /identity/v3/projects/8874969c142e4da89a95e46d23e55526/users/22f645a9d4f8477eac627c5202e91e10/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:38.659453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.659812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.660160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.691422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] MFA Rules not processed for user `22f645a9d4f8477eac627c5202e91e10`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:38.692652 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:38.771778 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1323/2653] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:50:38.775533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.820584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:38.858955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958261-1d16-483e-860b-9a7dacf52449 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.863893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.864491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.864768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.865054 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.865728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.867730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:38.897501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1331/2654] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:38.902658 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:38.914126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c66fee-23f7-4fba-bfdd-832310b94250 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:38.976748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:38.977468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:38.977746 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:38.978019 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:38.978688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:38.981361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.008694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1324/2655] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:38 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:39.013420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.027215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7187e-245b-4626-8b2e-23cc9a509c84 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.031880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.032491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.032793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.033053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.033692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.035321 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.046857 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1332/2656] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:39.051005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.062180 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434830d5-e96b-4620-a00f-e6418906aab0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.067671 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.068464 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.068814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.069098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] PATH_INFO: `/v3/projects/90169a62e55c4b2084964c1317c22f15/users/75ebcae863224b3d963ebec944307040/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.087801 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90169a62e55c4b2084964c1317c22f15, user_id=75ebcae863224b3d963ebec944307040, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.091355 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.095213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:39.114271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1325/2657] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:39 2022] PUT /identity/v3/projects/90169a62e55c4b2084964c1317c22f15/users/75ebcae863224b3d963ebec944307040/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:39.116257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.127628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6cd2-5e04-4593-92db-d86b240e4165 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.186188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.186854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.187140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.187419 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.188072 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.189770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb724cf-3003-4412-b629-3a40e8c36f5a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.201884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1333/2658] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:50:39.206403 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.217765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd2308-8c7f-4af3-b9e1-f636639df141 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.222491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.223142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.223414 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.223685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] PATH_INFO: `/v3/projects/90169a62e55c4b2084964c1317c22f15/users/75ebcae863224b3d963ebec944307040/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.225591 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90169a62e55c4b2084964c1317c22f15, user_id=75ebcae863224b3d963ebec944307040, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.228307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.238787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1326/2659] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:39 2022] PUT /identity/v3/projects/90169a62e55c4b2084964c1317c22f15/users/75ebcae863224b3d963ebec944307040/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:39.250335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.250692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.250973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.281036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] MFA Rules not processed for user `75ebcae863224b3d963ebec944307040`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:39.282171 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:39.350612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1334/2660] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:39.361913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.363858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:39.433339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34181fd3-f205-4c67-9ff8-01745f6da50f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.438378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.439080 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.439472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.439791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.452014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.453451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58856f-9748-4595-bef8-152452b00fd4 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.463401 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1327/2661] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:39.496351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.523035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:39.558204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a2c63-6ba8-43cb-90d1-8acf58b34eb5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.562615 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.563237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.563517 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.563789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] PATH_INFO: `/v3/users/22f645a9d4f8477eac627c5202e91e10` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.564898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f645a9d4f8477eac627c5202e91e10)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.566862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.613083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f645a9d4f8477eac627c5202e91e10'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:39.621080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f645a9d4f8477eac627c5202e91e10'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:39.621865 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f645a9d4f8477eac627c5202e91e10'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:39.622524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f645a9d4f8477eac627c5202e91e10'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:39.630182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f645a9d4f8477eac627c5202e91e10'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:39.647978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1335/2662] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:39 2022] DELETE /identity/v3/users/22f645a9d4f8477eac627c5202e91e10 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:39.652717 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.653005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.653283 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.671740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:39.672934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:39.731311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1328/2663] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:39.742379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:39.743842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:39.811119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819758-1836-4454-8852-74a14a3ac8b6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:39.815045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:39.815752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:39.816109 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:39.816429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:39.825722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:39.827184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:39.836508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1336/2664] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:40.111458 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:40.113296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:40.128318 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfafb5d-d5c3-4f09-872f-2fd28bd68a67 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:40.133688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:40.134326 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.134810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.135103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] PATH_INFO: `/v3/projects/8874969c142e4da89a95e46d23e55526` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.136319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8874969c142e4da89a95e46d23e55526)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:40.138229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:40.186519 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8874969c142e4da89a95e46d23e55526', 'request_id': 'req-54da3ead-fd61-4c1a-946d-8549bb588cfe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.193981 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8874969c142e4da89a95e46d23e55526', 'request_id': 'req-54da3ead-fd61-4c1a-946d-8549bb588cfe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.200503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8874969c142e4da89a95e46d23e55526', 'request_id': 'req-54da3ead-fd61-4c1a-946d-8549bb588cfe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.210753 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] The token cache is being invalidate because project 8874969c142e4da89a95e46d23e55526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:40.211354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.211670 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.213136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1329/2665] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:40 2022] DELETE /identity/v3/projects/8874969c142e4da89a95e46d23e55526 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:40.217735 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:40.227309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38a09c-344a-437e-99d1-b791eef3bd00 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:40.271799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:40.287212 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:40.287851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.288130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.288406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] PATH_INFO: `/v3/users/75ebcae863224b3d963ebec944307040` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.289519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ebcae863224b3d963ebec944307040)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:40.291501 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:40.340172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ebcae863224b3d963ebec944307040'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.347034 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ebcae863224b3d963ebec944307040'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.347703 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ebcae863224b3d963ebec944307040'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.348308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ebcae863224b3d963ebec944307040'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.356326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b07e3224-98c5-4144-9e29-01869f51bd85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ebcae863224b3d963ebec944307040'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.374071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1337/2666] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:40 2022] DELETE /identity/v3/users/75ebcae863224b3d963ebec944307040 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:40.664534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:40.665991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:40.680620 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da3ead-fd61-4c1a-946d-8549bb588cfe admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:40.685607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:40.742888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:40.743515 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.743796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.744123 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] PATH_INFO: `/v3/projects/90169a62e55c4b2084964c1317c22f15` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.745399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90169a62e55c4b2084964c1317c22f15)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:40.747687 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:40.798110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90169a62e55c4b2084964c1317c22f15', 'request_id': 'req-189618c8-3c34-456d-8b2d-22483a4ea80d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.805678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90169a62e55c4b2084964c1317c22f15', 'request_id': 'req-189618c8-3c34-456d-8b2d-22483a4ea80d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.812373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90169a62e55c4b2084964c1317c22f15', 'request_id': 'req-189618c8-3c34-456d-8b2d-22483a4ea80d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.824076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] The token cache is being invalidate because project 90169a62e55c4b2084964c1317c22f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:40.824377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.825052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-189618c8-3c34-456d-8b2d-22483a4ea80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:40.826557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1330/2667] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:40 2022] DELETE /identity/v3/projects/90169a62e55c4b2084964c1317c22f15 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:40.831439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.831730 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.832010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.852124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:40.853328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:40.914584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1338/2668] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:40.922311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.922618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.922896 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.944061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:40.945266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:40.954174 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1331/2669] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:40.958602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:40.968292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5949ef-9dea-4385-b07f-7e4be7712f32 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:40.972872 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:40.973467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:40.973750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:40.974027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:40.974706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:40.975013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:40.976613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:40.988095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1339/2670] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:40.992439 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.002241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ff945-1cd6-4ba7-b884-e1a4cce537d6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.007402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.008017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.008294 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.008560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.009197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.011691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.037442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1332/2671] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:40 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:41.041609 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.052035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47fadb-6631-46ce-914f-392d98c02eb2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.101010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:41.112324 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.113126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.113420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.113859 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.114538 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.116468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.137801 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1340/2672] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:41.141951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.152743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd974-ebbd-4464-9855-82b222e2d85e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.157281 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.157866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.158143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.158430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.159068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.160646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.171450 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1333/2673] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:41.175277 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.185370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7992-a9fd-465b-b1d7-cfdb6d9c94cb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.190112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.190736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.191013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.191283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.191915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.193485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176ae69-b8fd-4c71-98a0-156794d40dbb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.204078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1341/2674] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:50:41.208071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.218329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ad8f-dc01-454f-94ad-bb94204f9dee admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.223236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.223937 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.224223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.224607 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] PATH_INFO: `/v3/projects/58d16f44a51e4912ac60fe95c88853f3/users/4aac2f65f0d34364bb79103225c57bcd/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.240572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d16f44a51e4912ac60fe95c88853f3, user_id=4aac2f65f0d34364bb79103225c57bcd, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.243038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.256314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1334/2675] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:41 2022] PUT /identity/v3/projects/58d16f44a51e4912ac60fe95c88853f3/users/4aac2f65f0d34364bb79103225c57bcd/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:41.267541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.268013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.268147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.296187 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] MFA Rules not processed for user `4aac2f65f0d34364bb79103225c57bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:41.297406 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:41.362846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1342/2676] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:41.367373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.405764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:41.429612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a645-6706-4a98-94c0-ea8af2ff1992 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.434274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.435085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.435503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.435954 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.436938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.440056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.470431 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1335/2677] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:41.474724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.485657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78a844-908e-43d1-b999-7d5bbf3d6128 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.541365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.542003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.542285 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.542591 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.543220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.545137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.566084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1343/2678] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:41.570179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.580854 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92ccac-f7a8-4c50-bbab-e1c118f37980 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.585407 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.585991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.586263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.586565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.587197 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.589460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.599834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1336/2679] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:41.603918 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.614509 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bacd2a-0913-4e66-afff-36dcde8f1036 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.619168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.619767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.620571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.620915 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.621681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.623319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5af2fb-8a33-4efb-a558-829f232cf7de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.633249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1344/2680] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:41.638084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.647934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717f3e-e3b4-42a3-a2f1-276698199169 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.652461 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.653091 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.653371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.653651 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] PATH_INFO: `/v3/projects/086734e1f65b4f7c91b70957968f4ad6/users/0e948028d58d4d00ab7f9d17ac6ef8c8/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.668036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=086734e1f65b4f7c91b70957968f4ad6, user_id=0e948028d58d4d00ab7f9d17ac6ef8c8, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.670744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.684244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1337/2681] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:41 2022] PUT /identity/v3/projects/086734e1f65b4f7c91b70957968f4ad6/users/0e948028d58d4d00ab7f9d17ac6ef8c8/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:41.749943 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.750231 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.750532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.778595 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] MFA Rules not processed for user `0e948028d58d4d00ab7f9d17ac6ef8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:41.779800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:41.808476 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:41.844875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1345/2682] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:41.849711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.910511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a09dc-25a7-465c-b952-c1daa4d9de6b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:41.915190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:41.915793 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:41.916075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:41.916349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:41.917068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:41.919061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:41.946348 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1338/2683] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:41.951125 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:41.961764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d1ae-ece5-4395-aebf-869200fac14f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.018714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.019323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.019605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.019881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.020519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.022431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.043822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1346/2684] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:41 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:42.047792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.059060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0177b0a-3cd5-4836-8b29-a829881de221 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.063677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.064295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.064575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.064853 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.065492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.067113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.078132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1339/2685] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:42.082181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.092867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26442074-1efa-4a56-8275-14c2f64a0e16 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.097718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.098366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.098670 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.098941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] PATH_INFO: `/v3/projects/3e3b22ebbee243a1be6960cef2b9472f/users/b238472b54714b2da9050012cdcbc7ea/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.116879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b22ebbee243a1be6960cef2b9472f, user_id=b238472b54714b2da9050012cdcbc7ea, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.119622 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.135578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1347/2686] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:42 2022] PUT /identity/v3/projects/3e3b22ebbee243a1be6960cef2b9472f/users/b238472b54714b2da9050012cdcbc7ea/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:42.139512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.149884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a0685-7847-4707-ba7e-5c3eb35707fb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.206252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.206824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.207109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.207386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.208030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.209629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89f1e-e2f4-4b83-9f3f-9d9739c22fd2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.210873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:42.221345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1340/2687] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:42.225335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.236229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c337-0891-47db-9ee3-ee5448771bde admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.240868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.241513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.241791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.242062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] PATH_INFO: `/v3/projects/3e3b22ebbee243a1be6960cef2b9472f/users/b238472b54714b2da9050012cdcbc7ea/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.244014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b22ebbee243a1be6960cef2b9472f, user_id=b238472b54714b2da9050012cdcbc7ea, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.247275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.257724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1348/2688] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:42 2022] PUT /identity/v3/projects/3e3b22ebbee243a1be6960cef2b9472f/users/b238472b54714b2da9050012cdcbc7ea/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:42.268184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.268471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.268740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.296454 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] MFA Rules not processed for user `b238472b54714b2da9050012cdcbc7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:42.297653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:42.364856 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1341/2689] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:42.375706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.377425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:42.439876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf44984-149b-4ce8-86ee-ec043d4d4ddf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.443678 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.444372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.444693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.445003 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:42.454726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:42.456161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:42.466185 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1349/2690] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:42.970976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:42.972370 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:42.984779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b74aa2-4bb6-400d-87a3-71e6fb9d840f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:42.988233 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:42.988820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:42.989095 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:42.989362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:43.046638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:43.048060 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:43.057160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1342/2691] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:47.090688 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:47.109823 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857463-5b46-4678-ad13-28d010487b67 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:47.113422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:47.114016 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:47.114295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:47.114603 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:47.124354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:47.187632 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:47.189119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e68b-cbd8-471c-8285-16bb353da379 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:47.198025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1350/2692] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:49.298653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:49.376411 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:49.403361 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3aa9b-0da0-4a36-9f29-a5d350472a17 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:49.407980 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:49.408595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:49.408874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:49.409148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] PATH_INFO: `/v3/users/4aac2f65f0d34364bb79103225c57bcd` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:49.410260 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aac2f65f0d34364bb79103225c57bcd)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:49.412188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:49.458575 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aac2f65f0d34364bb79103225c57bcd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:49.465408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aac2f65f0d34364bb79103225c57bcd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:49.466110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aac2f65f0d34364bb79103225c57bcd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:49.466839 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aac2f65f0d34364bb79103225c57bcd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:49.474252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aac2f65f0d34364bb79103225c57bcd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:49.492669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1343/2693] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:49 2022] DELETE /identity/v3/users/4aac2f65f0d34364bb79103225c57bcd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:49.497788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:49.498275 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:49.498382 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:49.547010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:49.548235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:49.606132 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1351/2694] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:49.616268 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:49.617742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:49.685285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa43e9-e3b7-495a-9195-5967c9d2ef0f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:49.689105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:49.689699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:49.689978 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:49.690249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:49.699087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:49.700521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:49.709814 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1344/2695] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:49.990294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:49.991740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:50.005835 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96509fd9-8b50-44c2-8d22-46db45f6855b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:50.010886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:50.011524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:50.011898 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:50.012174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] PATH_INFO: `/v3/projects/58d16f44a51e4912ac60fe95c88853f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:50.013333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d16f44a51e4912ac60fe95c88853f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:50.015400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:50.064174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d16f44a51e4912ac60fe95c88853f3', 'request_id': 'req-b820e658-960b-4075-bd9b-1c49499ede63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.069568 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d16f44a51e4912ac60fe95c88853f3', 'request_id': 'req-b820e658-960b-4075-bd9b-1c49499ede63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.076744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d16f44a51e4912ac60fe95c88853f3', 'request_id': 'req-b820e658-960b-4075-bd9b-1c49499ede63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.078710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:50.090806 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] The token cache is being invalidate because project 58d16f44a51e4912ac60fe95c88853f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:50.091096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.091758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.093259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1352/2696] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:49 2022] DELETE /identity/v3/projects/58d16f44a51e4912ac60fe95c88853f3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:50.097864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:50.108154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2fb8-0f9d-42f6-9ca1-87ef57dedce7 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:50.156459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:50.168618 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:50.169234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:50.169508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:50.169771 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] PATH_INFO: `/v3/users/0e948028d58d4d00ab7f9d17ac6ef8c8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:50.170904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e948028d58d4d00ab7f9d17ac6ef8c8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:50.172813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:50.218253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e948028d58d4d00ab7f9d17ac6ef8c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.225321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e948028d58d4d00ab7f9d17ac6ef8c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.225987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e948028d58d4d00ab7f9d17ac6ef8c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.226636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e948028d58d4d00ab7f9d17ac6ef8c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.233862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e948028d58d4d00ab7f9d17ac6ef8c8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.251949 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1345/2697] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:50 2022] DELETE /identity/v3/users/0e948028d58d4d00ab7f9d17ac6ef8c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:50.528690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:50.530186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:50.544252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820e658-960b-4075-bd9b-1c49499ede63 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:50.604491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:50.605213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:50.605486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:50.605773 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] PATH_INFO: `/v3/projects/086734e1f65b4f7c91b70957968f4ad6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:50.606947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=086734e1f65b4f7c91b70957968f4ad6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:50.608883 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:50.657793 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086734e1f65b4f7c91b70957968f4ad6', 'request_id': 'req-42aa035a-4a3e-4040-b7ad-c20981faf18e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.663159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086734e1f65b4f7c91b70957968f4ad6', 'request_id': 'req-42aa035a-4a3e-4040-b7ad-c20981faf18e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.670302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086734e1f65b4f7c91b70957968f4ad6', 'request_id': 'req-42aa035a-4a3e-4040-b7ad-c20981faf18e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.681592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] The token cache is being invalidate because project 086734e1f65b4f7c91b70957968f4ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:50.681878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.682571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.684573 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1353/2698] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:50 2022] DELETE /identity/v3/projects/086734e1f65b4f7c91b70957968f4ad6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:50.688967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:50.690751 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:50.704308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fce9e-7c49-4ed8-a656-049cd25f87a8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:50.761350 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:50.762772 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:50.763052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:50.763328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] PATH_INFO: `/v3/users/b238472b54714b2da9050012cdcbc7ea` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:50.764475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b238472b54714b2da9050012cdcbc7ea)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:50.766395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:50.817154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b238472b54714b2da9050012cdcbc7ea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.825471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b238472b54714b2da9050012cdcbc7ea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.826337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b238472b54714b2da9050012cdcbc7ea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.826972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b238472b54714b2da9050012cdcbc7ea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.834247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3faf050b-8839-451d-9765-09551fb4d2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b238472b54714b2da9050012cdcbc7ea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:50.841253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:50.854684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1346/2699] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:50 2022] DELETE /identity/v3/users/b238472b54714b2da9050012cdcbc7ea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:51.121916 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.123999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:51.142667 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa035a-4a3e-4040-b7ad-c20981faf18e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.198871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.199524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.199805 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.200078 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] PATH_INFO: `/v3/projects/3e3b22ebbee243a1be6960cef2b9472f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.201182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3b22ebbee243a1be6960cef2b9472f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.203107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.233930 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:51.250590 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3b22ebbee243a1be6960cef2b9472f', 'request_id': 'req-965cb495-ad12-4887-8675-4e6ebf0f5d25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:51.256238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3b22ebbee243a1be6960cef2b9472f', 'request_id': 'req-965cb495-ad12-4887-8675-4e6ebf0f5d25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:51.263309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3b22ebbee243a1be6960cef2b9472f', 'request_id': 'req-965cb495-ad12-4887-8675-4e6ebf0f5d25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:51.275064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] The token cache is being invalidate because project 3e3b22ebbee243a1be6960cef2b9472f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:51.276149 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:51.276988 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-965cb495-ad12-4887-8675-4e6ebf0f5d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:51.278588 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1354/2700] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:51 2022] DELETE /identity/v3/projects/3e3b22ebbee243a1be6960cef2b9472f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:51.284773 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.285156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.285533 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.336016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:51.337213 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:51.395924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1347/2701] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:51.406099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.406384 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.406698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.427524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:51.428982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:51.437083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1355/2702] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:51.441373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.451231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec9b84-3801-4aa6-b2f2-cf86cbd4c5e0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.455794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.456386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.456669 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.456944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.457616 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:51.457923 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.459587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.471982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1348/2703] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:51.476077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.485975 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7019c-3e87-41d5-bfca-17c4ebcbe154 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.490650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.491241 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.491519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.491790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.492427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.494318 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.520587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1356/2704] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:51 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:51.524812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.535608 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35f5c1-fa46-4d98-bc3f-a3a736bb2b0e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.594298 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.594945 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.595231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.595503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.596134 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.598200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.624779 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1349/2705] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:51 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:51.626056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.636482 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb9d63-27d4-4392-b777-3267f2bd8a0e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.641172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.641767 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.642043 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.642311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.642981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.644589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.655084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1357/2706] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:51.659070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.669720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3bef5-6832-4c02-bca7-f76e833738c2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.674357 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.675008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.675337 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.675557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.676202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.677961 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51638-5d67-4556-9b39-aa02a726eadf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.689075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1350/2707] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:51.693274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.704015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc6ff-94a6-45a8-9a95-a96a210cbe41 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.708640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.709315 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.709597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.709870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] PATH_INFO: `/v3/projects/9bb94b9430fa47aaa73825d9f3f1ca61/users/3cad786cd51d46afa09eb72fc0f4e33a/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.724928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb94b9430fa47aaa73825d9f3f1ca61, user_id=3cad786cd51d46afa09eb72fc0f4e33a, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.727648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.742067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1358/2708] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:51 2022] PUT /identity/v3/projects/9bb94b9430fa47aaa73825d9f3f1ca61/users/3cad786cd51d46afa09eb72fc0f4e33a/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:51.754210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.754210 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.754452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.783343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] MFA Rules not processed for user `3cad786cd51d46afa09eb72fc0f4e33a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:51.784587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:51.803834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:51.855053 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1351/2709] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:51.866648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:51.868602 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:51.928032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd8255-e74f-437d-a832-e599b7895a55 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:51.931841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:51.932443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:51.932726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:51.932999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:51.941180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:51.942634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7503c-d15a-4f21-b8c8-e5115fda50c2 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:51.952076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1359/2710] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:01.852248 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=18 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:01.947480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:14.706440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:14.767424 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c85e32-268f-4b92-994f-06a47b776bc7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:14.771850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:14.772468 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:14.772749 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:14.773022 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] PATH_INFO: `/v3/users/3cad786cd51d46afa09eb72fc0f4e33a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:14.774126 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cad786cd51d46afa09eb72fc0f4e33a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:14.776078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:14.821860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cad786cd51d46afa09eb72fc0f4e33a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:14.828932 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cad786cd51d46afa09eb72fc0f4e33a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:14.829646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cad786cd51d46afa09eb72fc0f4e33a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:14.830209 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cad786cd51d46afa09eb72fc0f4e33a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:14.838345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cad786cd51d46afa09eb72fc0f4e33a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:14.856934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1352/2711] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:14 2022] DELETE /identity/v3/users/3cad786cd51d46afa09eb72fc0f4e33a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:14.861604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:14.862122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:14.862217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:14.882169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:14.883448 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:14.941914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1360/2712] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:51:14.952787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:14.954214 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:15.035002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950bca2-0bfa-4d9f-8f39-c461d7b60065 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.038713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.039313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.039595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.039870 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.048831 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.050259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad3f92-8bf6-4cf4-88c6-2b0158c99811 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.059938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1353/2713] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:15.339613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.341577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:15.353453 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676728d4-2510-4257-b2d2-6a19faec05b6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.358998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.359630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.359912 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.360215 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] PATH_INFO: `/v3/projects/9bb94b9430fa47aaa73825d9f3f1ca61` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.361401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb94b9430fa47aaa73825d9f3f1ca61)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.363378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.411880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb94b9430fa47aaa73825d9f3f1ca61', 'request_id': 'req-86969e18-3a11-4467-abe6-86c666342544'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:15.417367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb94b9430fa47aaa73825d9f3f1ca61', 'request_id': 'req-86969e18-3a11-4467-abe6-86c666342544'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:15.430203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb94b9430fa47aaa73825d9f3f1ca61', 'request_id': 'req-86969e18-3a11-4467-abe6-86c666342544'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:15.442432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] The token cache is being invalidate because project 9bb94b9430fa47aaa73825d9f3f1ca61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:15.442732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:15.443364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-86969e18-3a11-4467-abe6-86c666342544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:15.444847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1361/2714] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:15 2022] DELETE /identity/v3/projects/9bb94b9430fa47aaa73825d9f3f1ca61 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:15.449872 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.450158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.450447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.458015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:15.471873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:15.473083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:15.532096 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1354/2715] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:51:15.540897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.541184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.541500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.562940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:15.564213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:15.572053 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1362/2716] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:51:15.576274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.585930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183be3a-68a6-4d79-bfe5-6f711b07ac46 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.591030 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.591634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.591904 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.592179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.592826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:51:15.593132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.594776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.607013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1355/2717] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:51:15.611122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.620176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b552dd9-b5cf-4712-bc4f-ca3a100deadd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.624860 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.625443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.625716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.626013 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.626657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.628514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.657104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1363/2718] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:51:15 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:51:15.661485 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.671536 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c2c08-e86f-4b22-b7ed-ac3e7a8c9232 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.726252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.727000 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.727251 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.727534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.728154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.730092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.750639 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1356/2719] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:51:15 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:51:15.754781 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.765504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745f7d-e276-45c0-a302-6ac5901d3a78 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.770118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.770756 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.771035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.771309 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.771947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.773549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.784702 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1364/2720] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:51:15.788706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.798667 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dc172-ad50-4a38-936f-787a2d05d678 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.803489 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.804075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.804385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.804655 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.805292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.806917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ef5b2-1d70-4d15-a2b5-5a8a69287cb5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.817341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1357/2721] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:51:15.821594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.831838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde54a-2812-4e74-8c73-6458394867f1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:15.836439 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:15.837083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.837368 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.837642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] PATH_INFO: `/v3/projects/728f0932b2114572854db55c2694106a/users/10b5766198024e2194f810f5358de078/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.852940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] RBAC: Authorizing `identity:create_grant(project_id=728f0932b2114572854db55c2694106a, user_id=10b5766198024e2194f810f5358de078, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:15.855629 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:15.868123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1365/2722] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:51:15 2022] PUT /identity/v3/projects/728f0932b2114572854db55c2694106a/users/10b5766198024e2194f810f5358de078/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:15.878719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:15.879009 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:15.879311 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:15.908629 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] MFA Rules not processed for user `10b5766198024e2194f810f5358de078`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:15.909824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:15.968497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:15.974038 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1358/2723] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:51:15.984734 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:15.986390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:16.008929 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:16.064977 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860820aa-3573-492b-ae54-dd471d4f653a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:16.068789 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:16.069392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:16.069674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:16.069948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:16.078840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:16.080253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:16.089148 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1366/2724] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:18.412840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:18.414264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:18.471610 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e83e-b89f-4001-b3b8-a3f844b86936 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:18.475184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:18.475789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:18.476068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:18.476341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:18.478052 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:18.537015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:18.538365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:18.547944 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1359/2725] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:22.787722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:22.789196 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:22.880232 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d6c-b88f-49f8-8a78-5ee959bd3a7b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:22.885152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:22.885753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:22.886033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:22.886310 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:22.888608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:22.900238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:22.901792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:22.911554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1367/2726] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:24.761194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:24.770207 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87d14-fd50-4e95-b914-d1195d71c8b5 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:24.773812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:24.774406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:24.774712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:24.774986 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:24.776742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:24.838858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:24.840277 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a42041-a56b-4897-994f-281fd7c34893 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:24.849989 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1360/2727] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:26.065817 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:26.078693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d399b2-e500-406c-9ee5-5fe9d7b7e0f7 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:26.083836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:26.084594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:26.084933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:26.085237 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] PATH_INFO: `/v3/users/10b5766198024e2194f810f5358de078` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:26.086364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b5766198024e2194f810f5358de078)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:26.088353 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:26.138077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b5766198024e2194f810f5358de078'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.144940 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b5766198024e2194f810f5358de078'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.145610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b5766198024e2194f810f5358de078'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.146207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b5766198024e2194f810f5358de078'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.153764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b5766198024e2194f810f5358de078'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.158659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:26.176854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1368/2728] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:26 2022] DELETE /identity/v3/users/10b5766198024e2194f810f5358de078 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:26.181605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:26.182088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:26.182195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:26.201905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:26.203149 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:26.244525 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:26.283857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1361/2729] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:51:26.294809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:26.296262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:26.370086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6988221-24dd-481b-b419-422ffdcf5763 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:26.374878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:26.375737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:26.377334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:26.377711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:26.388364 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:26.389896 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:26.401370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1369/2730] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:26.686735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:26.688890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:26.703520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0fe67-8a12-4678-b902-63a3475c702a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:26.716768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:26.717550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:26.717905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:26.718252 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] PATH_INFO: `/v3/projects/728f0932b2114572854db55c2694106a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:26.719730 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=728f0932b2114572854db55c2694106a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:26.722317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:26.784025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728f0932b2114572854db55c2694106a', 'request_id': 'req-1a68044f-7f32-464b-9720-ddb8ea1cf48f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.792181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728f0932b2114572854db55c2694106a', 'request_id': 'req-1a68044f-7f32-464b-9720-ddb8ea1cf48f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.799878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728f0932b2114572854db55c2694106a', 'request_id': 'req-1a68044f-7f32-464b-9720-ddb8ea1cf48f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.812593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] The token cache is being invalidate because project 728f0932b2114572854db55c2694106a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:26.812898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.813572 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:26.815342 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1362/2731] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:26 2022] DELETE /identity/v3/projects/728f0932b2114572854db55c2694106a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:31.074569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:31.076038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:31.094932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9519f-b9b0-4d08-913f-f86b1738b4b9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:31.155743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:31.156366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:31.156641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:31.156911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] PATH_INFO: `/v3/users/0ca903cc98a24621b7c3be488e3e7c5e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:31.158015 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca903cc98a24621b7c3be488e3e7c5e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:31.160009 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:31.205151 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca903cc98a24621b7c3be488e3e7c5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.211828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca903cc98a24621b7c3be488e3e7c5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.212494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca903cc98a24621b7c3be488e3e7c5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.213068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca903cc98a24621b7c3be488e3e7c5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.219828 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca903cc98a24621b7c3be488e3e7c5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.236944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1370/2732] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:31 2022] DELETE /identity/v3/users/0ca903cc98a24621b7c3be488e3e7c5e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:31.410135 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:31.411684 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:31.428339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68044f-7f32-464b-9720-ddb8ea1cf48f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:31.485738 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:31.486368 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:31.486718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:31.486999 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] PATH_INFO: `/v3/projects/b62283fd6a124de0a32ad8f586cf997e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:31.488117 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62283fd6a124de0a32ad8f586cf997e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:31.490064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:31.535690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62283fd6a124de0a32ad8f586cf997e', 'request_id': 'req-ce8ca313-2529-4cb2-99e6-5e72159b078f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.542270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62283fd6a124de0a32ad8f586cf997e', 'request_id': 'req-ce8ca313-2529-4cb2-99e6-5e72159b078f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.549215 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62283fd6a124de0a32ad8f586cf997e', 'request_id': 'req-ce8ca313-2529-4cb2-99e6-5e72159b078f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.556642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:31.561595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] The token cache is being invalidate because project b62283fd6a124de0a32ad8f586cf997e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:31.562102 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.563224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.564818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1363/2733] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:31 2022] DELETE /identity/v3/projects/b62283fd6a124de0a32ad8f586cf997e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:31.568842 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:31.570392 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:31.583462 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097698d-a525-4d52-8c0c-65741e819892 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:31.587074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:31.641740 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:31.642370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:31.642681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:31.642959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] PATH_INFO: `/v3/users/3db3c6d0b14a497d9aeee6ea7ab09546` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:31.644063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db3c6d0b14a497d9aeee6ea7ab09546)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:31.645969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:31.691243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db3c6d0b14a497d9aeee6ea7ab09546'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.698024 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db3c6d0b14a497d9aeee6ea7ab09546'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.699210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db3c6d0b14a497d9aeee6ea7ab09546'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.699816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db3c6d0b14a497d9aeee6ea7ab09546'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.707714 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a8642469-f847-4409-8134-26f108ab53b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db3c6d0b14a497d9aeee6ea7ab09546'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:31.725757 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1371/2734] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:31 2022] DELETE /identity/v3/users/3db3c6d0b14a497d9aeee6ea7ab09546 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:31.892220 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:31.893659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:31.907239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ca313-2529-4cb2-99e6-5e72159b078f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:31.965111 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:31.965739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:31.966026 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:31.966300 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] PATH_INFO: `/v3/projects/113b0f00d7b04db0b1f3ae9cba916275` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:31.967452 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=113b0f00d7b04db0b1f3ae9cba916275)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:31.969363 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:32.016543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113b0f00d7b04db0b1f3ae9cba916275', 'request_id': 'req-4c6c57de-71aa-468d-9022-0f351234b31c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:32.023700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113b0f00d7b04db0b1f3ae9cba916275', 'request_id': 'req-4c6c57de-71aa-468d-9022-0f351234b31c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:32.030686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113b0f00d7b04db0b1f3ae9cba916275', 'request_id': 'req-4c6c57de-71aa-468d-9022-0f351234b31c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:32.041049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] The token cache is being invalidate because project 113b0f00d7b04db0b1f3ae9cba916275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:32.041443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:32.042078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:32.044120 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1364/2735] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:31 2022] DELETE /identity/v3/projects/113b0f00d7b04db0b1f3ae9cba916275 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:41.722276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:42.037930 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:49.126760 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:49.127249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:49.127369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:49.158090 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:49.159334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:49.221674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1372/2736] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 15:51:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:51:49.268653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:49.270253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:49.338510 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c57de-71aa-468d-9022-0f351234b31c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:49.344175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:49.344785 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:49.345066 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:49.345340 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:49.353480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:49.355114 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:49.363087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1365/2737] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:51:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:51:52.733879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:52.735305 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:52.829952 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3cf4d-61b3-428c-9627-b2388d09edbf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:52.834939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:52.835551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:52.835827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:52.836097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:52.837847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:52.849796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:52.851394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:52.861866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1373/2738] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:56.640964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:56.642443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:56.707768 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffc1d8-9d61-4591-af0e-79603a546cfe service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:56.713374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:56.714008 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:56.714291 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:56.714606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] PATH_INFO: `/v3/users/525cdc42a473486ba64b481cf2820d31` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:56.715790 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525cdc42a473486ba64b481cf2820d31)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:56.718198 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:56.765217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525cdc42a473486ba64b481cf2820d31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:56.772062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525cdc42a473486ba64b481cf2820d31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:56.772728 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525cdc42a473486ba64b481cf2820d31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:56.773335 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525cdc42a473486ba64b481cf2820d31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:56.781232 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525cdc42a473486ba64b481cf2820d31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:56.799217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1366/2739] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:56 2022] DELETE /identity/v3/users/525cdc42a473486ba64b481cf2820d31 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:56.985418 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:57.003055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a9d-e97e-4d41-8169-013c13e0a1a2 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:57.083037 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:57.083669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:57.083981 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:57.084214 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] PATH_INFO: `/v3/projects/f3eec134d5864a3c82f27706eb8065fa` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:57.085345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3eec134d5864a3c82f27706eb8065fa)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:57.087268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:57.115832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:57.151010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3eec134d5864a3c82f27706eb8065fa', 'request_id': 'req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.158330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3eec134d5864a3c82f27706eb8065fa', 'request_id': 'req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.166949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3eec134d5864a3c82f27706eb8065fa', 'request_id': 'req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.179210 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] The token cache is being invalidate because project f3eec134d5864a3c82f27706eb8065fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:57.179499 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.180157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.181642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1374/2740] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:56 2022] DELETE /identity/v3/projects/f3eec134d5864a3c82f27706eb8065fa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:57.185692 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:57.187165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:57.200888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64300f5-52be-447b-b779-ad39a4beb567 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:57.206440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:57.273812 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:57.274585 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:57.274877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:57.275156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] PATH_INFO: `/v3/users/dcdf8ff919a24ed4824549e460778de4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:57.276321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdf8ff919a24ed4824549e460778de4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:57.283660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:57.333720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdf8ff919a24ed4824549e460778de4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.340566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdf8ff919a24ed4824549e460778de4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.341257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdf8ff919a24ed4824549e460778de4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.341860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdf8ff919a24ed4824549e460778de4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.349190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-187ff86c-7f66-41f8-878a-946a18833619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdf8ff919a24ed4824549e460778de4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.366263 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1367/2741] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:57 2022] DELETE /identity/v3/users/dcdf8ff919a24ed4824549e460778de4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:57.547628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:57.549483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:57.563438 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82916e5d-58d3-4922-b49e-2b7a7ea85fbf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:57.637639 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:57.638369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:57.638697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:57.638973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] PATH_INFO: `/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:57.640169 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8574fd3e7f214ed18d7e378ac96fbd40)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:57.642126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:57.692956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8574fd3e7f214ed18d7e378ac96fbd40', 'request_id': 'req-622ea50b-0b9e-4cae-99a6-739c9ce97972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.699396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8574fd3e7f214ed18d7e378ac96fbd40', 'request_id': 'req-622ea50b-0b9e-4cae-99a6-739c9ce97972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.708973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8574fd3e7f214ed18d7e378ac96fbd40', 'request_id': 'req-622ea50b-0b9e-4cae-99a6-739c9ce97972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.722919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] The token cache is being invalidate because project 8574fd3e7f214ed18d7e378ac96fbd40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:57.723213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.723866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:57.725551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1375/2742] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:57 2022] DELETE /identity/v3/projects/8574fd3e7f214ed18d7e378ac96fbd40 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:52:07.362813 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:07.719518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:20.965154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866571-a59c-4229-9edc-19f87a1dd384 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:20.965664 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866571-a59c-4229-9edc-19f87a1dd384 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:20.965775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866571-a59c-4229-9edc-19f87a1dd384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:20.986509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fc866571-a59c-4229-9edc-19f87a1dd384 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:52:20.988862 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc866571-a59c-4229-9edc-19f87a1dd384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:21.048235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1368/2743] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:52:21.061124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:21.062644 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:21.130643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ea50b-0b9e-4cae-99a6-739c9ce97972 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:21.135844 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:21.136506 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:21.136809 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:21.137107 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:21.145452 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:21.147091 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:21.156335 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1376/2744] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:52:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:52:31.040169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=25 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:31.153297 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=32 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:37.463909 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:37.464412 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:37.464512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:37.484470 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:52:37.485598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:37.492756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1369/2745] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 15:52:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:52:37.504795 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:37.516390 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3e9b-a5a6-497f-9b3d-1b72ff2823bb service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:37.521400 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:37.522028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:37.522257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:37.522562 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:37.530997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:37.532546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:37.540677 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1377/2746] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:52:42.979783 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:42.981886 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:43.070377 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0ce26-6039-4559-a25f-435c1a6d112d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:43.074143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:43.074795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:43.075074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:43.075345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:43.079491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:43.170164 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:43.171581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:43.194265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1370/2747] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:46.772040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:46.781463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6195ad5-511d-47d7-842a-34cdb729cfb7 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:46.785021 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:46.785605 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:46.785884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:46.786155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:46.787867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:46.801084 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:46.802779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2678d4-38e2-4d7b-96dc-9df130e5999b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:46.814391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1378/2748] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:49.221308 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:49.222859 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:49.294791 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b290ee-999c-4cbb-a688-30d41e6dd9c7 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:49.298295 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:49.298920 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:49.299192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:49.299478 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:49.301160 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:49.333211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:49.362051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:49.363532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:49.372520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1371/2749] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:56.808979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=12 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:59.007580 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965c0ac-163d-414f-a418-c28041bed139 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:59.008182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965c0ac-163d-414f-a418-c28041bed139 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:59.008182 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965c0ac-163d-414f-a418-c28041bed139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:59.029248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7965c0ac-163d-414f-a418-c28041bed139 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:52:59.030503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7965c0ac-163d-414f-a418-c28041bed139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:59.037552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1379/2750] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 15:52:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:52:59.061349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:59.073075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8798-c43d-43a3-8f90-671b658b1b7e service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:59.078218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:59.078806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:59.079087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:59.079362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:59.087755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:59.089255 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:59.097474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1372/2751] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 15:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:53:00.428086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:00.428618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:00.428618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:00.447812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] MFA Rules not processed for user `88f34e4fc44e432d9e994ae5dcdbd1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:53:00.449007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:00.456548 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1380/2752] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:53:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:53:09.094163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=20 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:10.442518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:56.809202 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:56.818695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2341b55-2bf3-45ea-831e-11b7b4263204 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:56.822463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:56.823058 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:56.823338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:56.823606 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:56.825339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:56.835735 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:56.837219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:56.846373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1373/2753] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:53:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:53:58.671826 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:58.673884 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:58.693253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c1828-cddc-4e12-8c1d-0aee3965b792 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:58.699374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:58.700228 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:58.700681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:58.701651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:58.703834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:58.722334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:58.724039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:58.735973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1381/2754] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:53:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:03.442146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:03.443628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:03.454129 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2886d3-2626-49be-8f33-f1c0fd5da17a service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:03.457868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:03.458513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:03.458794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:03.459068 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:03.460890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:03.536784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:03.538142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:03.549056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1374/2755] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:06.668951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:06.671130 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:06.690104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d519b7c-ce2f-4210-97b6-a4bcf2801507 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:06.696836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:06.697672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:06.698223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:06.698700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] PATH_INFO: `/v3/users/9ecdfa3c79f84cc29061dd5ebfa60101` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:06.699867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ecdfa3c79f84cc29061dd5ebfa60101)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:06.702700 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:06.774174 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ecdfa3c79f84cc29061dd5ebfa60101'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:06.786653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ecdfa3c79f84cc29061dd5ebfa60101'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:06.787387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ecdfa3c79f84cc29061dd5ebfa60101'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:06.788144 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ecdfa3c79f84cc29061dd5ebfa60101'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:06.795875 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22f855fb-c991-4eb3-814c-7a20d5ae243a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ecdfa3c79f84cc29061dd5ebfa60101'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:06.813766 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1382/2756] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:54:06 2022] DELETE /identity/v3/users/9ecdfa3c79f84cc29061dd5ebfa60101 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:06.992916 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.005512 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7b7f-5e88-4cfe-bb14-725707d872ca service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.074151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.074818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.075229 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.075514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] PATH_INFO: `/v3/projects/c7541b0dd8b14bf495094ddaa132bc2a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.077033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7541b0dd8b14bf495094ddaa132bc2a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.080088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.138708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7541b0dd8b14bf495094ddaa132bc2a', 'request_id': 'req-f8fc9673-0bab-4514-a526-b48d0658b616'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:07.146675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7541b0dd8b14bf495094ddaa132bc2a', 'request_id': 'req-f8fc9673-0bab-4514-a526-b48d0658b616'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:07.153748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7541b0dd8b14bf495094ddaa132bc2a', 'request_id': 'req-f8fc9673-0bab-4514-a526-b48d0658b616'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:07.165643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] The token cache is being invalidate because project c7541b0dd8b14bf495094ddaa132bc2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:54:07.165848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:07.166469 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f8fc9673-0bab-4514-a526-b48d0658b616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:07.168015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1375/2757] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:54:06 2022] DELETE /identity/v3/projects/c7541b0dd8b14bf495094ddaa132bc2a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:07.173404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.173755 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.174101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.195093 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:07.196295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:07.258030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1383/2758] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:07.265964 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.266322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.266701 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.287247 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:07.288457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:07.296951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1376/2759] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:07.301143 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.311126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05b189-fa39-48e2-95ce-4823bd7e30e0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.315800 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.316404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.316689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.316965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.317611 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:54:07.317917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.319547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.331074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1384/2760] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:54:07.335372 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.340927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:07.345447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d985-2d2c-4467-bd0d-5ef2eac77d01 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.351490 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.352341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.352696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.353151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.353987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.355982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.398733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1377/2761] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:54:07 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:54:07.403070 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.416221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5ad76-58ae-4f98-895f-1bab52920ce3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.461990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:07.476816 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.477425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.477711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.478031 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.478762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.480752 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.503204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1385/2762] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:54:07 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:54:07.507204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.517659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b8917-0b85-4a89-95ac-b941cf800ddd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.522272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.522907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.523186 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.523459 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.524150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.525777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.537488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1378/2763] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:54:07.541444 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.552056 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdc280-d79c-4ba7-b23c-841a5cb529a9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.557188 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.557897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.558180 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.558484 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.559165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.560764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d807a-2147-4103-bd4d-9e4454b353ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.571815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1386/2764] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:54:07.576216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.586605 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1557-47ac-4dcf-9104-08a9b3016d16 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.591284 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.592463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.592739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.593037 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] PATH_INFO: `/v3/projects/0891d5f021664ff6b39f555c63ee5e5e/users/dae4ea8a6f5b4fc686bb07d31090ba3b/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.608380 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0891d5f021664ff6b39f555c63ee5e5e, user_id=dae4ea8a6f5b4fc686bb07d31090ba3b, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.611062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.625016 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1379/2765] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:54:07 2022] PUT /identity/v3/projects/0891d5f021664ff6b39f555c63ee5e5e/users/dae4ea8a6f5b4fc686bb07d31090ba3b/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:07.630076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.630389 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.630694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.650351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:07.651868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:07.715331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1387/2766] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:07.728061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:07.729869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:07.788635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228e90-5518-4ba8-bb34-f261d1fbe085 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:07.792423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:07.793025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:07.793303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:07.793622 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:07.803168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:07.804877 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdcdb4-66fc-4746-9bc2-2a78dcb516f6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:07.815257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1380/2767] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:09.383811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:09.394693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeab36-c5f3-4f43-977a-711c2056e195 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:09.398925 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:09.399544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:09.399824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:09.400099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:09.401879 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:09.415086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:09.416673 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:09.427575 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1388/2768] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:13.009885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da851fe-c597-45c1-999b-d6f5fe0e308f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:13.010365 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da851fe-c597-45c1-999b-d6f5fe0e308f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:13.010497 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da851fe-c597-45c1-999b-d6f5fe0e308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:13.045546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6da851fe-c597-45c1-999b-d6f5fe0e308f None None] MFA Rules not processed for user `dae4ea8a6f5b4fc686bb07d31090ba3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:13.047224 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6da851fe-c597-45c1-999b-d6f5fe0e308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:13.153142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1381/2769] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:54:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:13.153732 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:13.158299 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:13.288246 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340027-6ee2-45dc-a9db-81f3a97644c1 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:13.295581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:13.296447 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:13.296843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:13.297220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:13.314349 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:13.316498 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:13.338942 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1389/2770] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:15.048918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:15.049480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:15.049480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:15.080036 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] MFA Rules not processed for user `dae4ea8a6f5b4fc686bb07d31090ba3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:15.081672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:15.113903 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1382/2771] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:16.532394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:16.533841 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:16.589325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:16.633394 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34e7a-910b-4b6c-8a99-576f16e0bf45 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:16.639783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:16.640672 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:16.640970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:16.641227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:16.643717 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:16.658924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:16.660886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:16.672972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1390/2772] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:17.650754 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:17.652354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:17.698633 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3ac2c-aa3b-410d-b47e-a1c9ccda3144 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:17.704212 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:17.704868 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:17.705154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:17.705432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] PATH_INFO: `/v3/users/88f34e4fc44e432d9e994ae5dcdbd1ae` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:17.706643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f34e4fc44e432d9e994ae5dcdbd1ae)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:17.709124 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:17.742592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:17.807106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f34e4fc44e432d9e994ae5dcdbd1ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:17.817823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f34e4fc44e432d9e994ae5dcdbd1ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:17.818632 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f34e4fc44e432d9e994ae5dcdbd1ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:17.819463 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f34e4fc44e432d9e994ae5dcdbd1ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:17.828861 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f34e4fc44e432d9e994ae5dcdbd1ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:17.867818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1383/2773] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:54:17 2022] DELETE /identity/v3/users/88f34e4fc44e432d9e994ae5dcdbd1ae => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:18.093243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:18.105928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a327-82e4-4614-88e2-e84bdbd92d09 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:18.180265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:18.180979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:18.181330 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:18.181674 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] PATH_INFO: `/v3/projects/bcffff710dd74209b30a1d02fb2bec46` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:18.182897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcffff710dd74209b30a1d02fb2bec46)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:18.184833 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:18.232658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcffff710dd74209b30a1d02fb2bec46', 'request_id': 'req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.238294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcffff710dd74209b30a1d02fb2bec46', 'request_id': 'req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.245098 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcffff710dd74209b30a1d02fb2bec46', 'request_id': 'req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.258461 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] The token cache is being invalidate because project bcffff710dd74209b30a1d02fb2bec46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:54:18.258764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.259450 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.261685 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1391/2774] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:54:18 2022] DELETE /identity/v3/projects/bcffff710dd74209b30a1d02fb2bec46 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:18.266913 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:18.268432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:18.281857 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2752-2983-4f48-8907-154c4c566c2e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:18.347317 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:18.347955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:18.348241 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:18.348520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] PATH_INFO: `/v3/users/7ed1eefc2212459fb0d0935542e9e3f4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:18.349694 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed1eefc2212459fb0d0935542e9e3f4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:18.351810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:18.403226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed1eefc2212459fb0d0935542e9e3f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.411359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed1eefc2212459fb0d0935542e9e3f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.412076 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed1eefc2212459fb0d0935542e9e3f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.412707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed1eefc2212459fb0d0935542e9e3f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.420157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed1eefc2212459fb0d0935542e9e3f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.438455 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1384/2775] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:54:18 2022] DELETE /identity/v3/users/7ed1eefc2212459fb0d0935542e9e3f4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:18.625919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:18.627645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:18.642514 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf8fe-7869-4ba5-b52a-ae6ac066b50c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:18.699308 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:18.708103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:18.708746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:18.709544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:18.710063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] PATH_INFO: `/v3/projects/48716a5b014047bfbef18ebab68e5cd4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:18.711932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=48716a5b014047bfbef18ebab68e5cd4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:18.713932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:18.764122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48716a5b014047bfbef18ebab68e5cd4', 'request_id': 'req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.771066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48716a5b014047bfbef18ebab68e5cd4', 'request_id': 'req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.782808 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48716a5b014047bfbef18ebab68e5cd4', 'request_id': 'req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.795373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] The token cache is being invalidate because project 48716a5b014047bfbef18ebab68e5cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:54:18.795791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.798836 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:54:18.800642 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1392/2776] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:54:18 2022] DELETE /identity/v3/projects/48716a5b014047bfbef18ebab68e5cd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:54:24.685083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:24.686798 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:24.706106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:24.763911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af2f83-7a8f-4f6d-a556-e5d0741a1a0c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:24.827440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:24.827959 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:24.828234 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:24.828502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:24.830200 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:24.842093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:24.843686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:24.853967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1385/2777] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:28.792026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:34.849569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=55 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:56:53.933965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:53.935426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:54.008074 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc101-4ded-47d8-9ae5-d0048718a2aa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:54.013901 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:54.014541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:54.014822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:54.015095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:54.017431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:54.029147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:54.030791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175729b2-32b6-47b2-ba9e-a5617d91f0a5 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:54.041565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1393/2778] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:56:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:57:04.037104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=32 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:58:04.100960 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:58:04.102486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:58:04.158392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c9f1-3059-494a-b1f8-5142efc859e5 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:58:04.162235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:58:04.162865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:58:04.163155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:58:04.163429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:58:04.165159 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:58:04.179280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:58:04.180707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:58:04.192532 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1386/2779] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:58:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:58:09.699759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:58:09.700291 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:58:09.700410 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:58:09.720722 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] MFA Rules not processed for user `dae4ea8a6f5b4fc686bb07d31090ba3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:58:09.722459 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:58:09.779414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1394/2780] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:58:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:58:10.085675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:58:10.094323 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16721-5efa-4e78-b0cf-35562d22acb0 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:58:10.098466 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:58:10.099057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:58:10.099341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:58:10.099617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:58:10.101353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:58:10.110946 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:58:10.113097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:58:10.121341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1387/2781] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:58:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:58:19.770259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:58:20.118374 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=32 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:59:03.291969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:03.301648 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d4f9a-9731-4c58-9213-66a1a5af6adb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:03.305207 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:03.305803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:03.306087 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:03.306359 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:03.308118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:03.317834 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:03.320225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:03.328743 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1395/2782] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:59:13.324813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:59:25.814017 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:25.816002 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:25.832962 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8308d-27bd-40f5-b1fd-4e0978837547 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:25.838442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:25.839045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:25.839328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:25.839679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:25.841389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:25.851628 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:25.853190 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:25.862025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1388/2783] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:59:35.858051 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:59:50.808941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:50.823671 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b872-a706-4d60-9cd5-e898a20506a9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:50.831134 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:50.831887 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:50.832641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:50.832947 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:50.834868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:50.853201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:50.854922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:50.884342 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1396/2784] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:59:55.680012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:55.681718 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:55.693725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f13105-64dd-4a92-9d1b-b8c36eafca77 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:55.697402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:55.697991 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:55.698305 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:55.698584 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:55.700249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:55.761711 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:55.763169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:55.772993 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1389/2785] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:59:58.865640 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:58.867222 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:58.903524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd73f7-7a83-440d-85e4-43485ff5f87c service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:58.909124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:58.909739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:58.910216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:58.910694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:58.912500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:58.937668 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:58.939344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:58.948085 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:58.949541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:58.965038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1397/2786] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:59:58.965378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91e3e-9bf2-4e98-a632-97637da00ca6 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:58.975146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:58.976568 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:58.977828 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:58.978334 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] PATH_INFO: `/v3/users/dae4ea8a6f5b4fc686bb07d31090ba3b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:58.980045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae4ea8a6f5b4fc686bb07d31090ba3b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:58.982223 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.050748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae4ea8a6f5b4fc686bb07d31090ba3b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.058228 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae4ea8a6f5b4fc686bb07d31090ba3b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.059025 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae4ea8a6f5b4fc686bb07d31090ba3b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.059660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae4ea8a6f5b4fc686bb07d31090ba3b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.071504 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a5c2ff2-b136-49d8-9e7e-39056fb1ccb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae4ea8a6f5b4fc686bb07d31090ba3b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.094758 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1390/2787] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 15:59:58 2022] DELETE /identity/v3/users/dae4ea8a6f5b4fc686bb07d31090ba3b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:59:59.286604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.304334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca287f45-4f96-4330-85bc-69d0e6e97b4a service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.377312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.377961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.378242 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.378552 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] PATH_INFO: `/v3/projects/0891d5f021664ff6b39f555c63ee5e5e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.379723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0891d5f021664ff6b39f555c63ee5e5e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.383257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.435378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0891d5f021664ff6b39f555c63ee5e5e', 'request_id': 'req-6abba7c1-6221-41e9-87cb-15541747f271'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.441204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0891d5f021664ff6b39f555c63ee5e5e', 'request_id': 'req-6abba7c1-6221-41e9-87cb-15541747f271'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.449317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0891d5f021664ff6b39f555c63ee5e5e', 'request_id': 'req-6abba7c1-6221-41e9-87cb-15541747f271'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.460564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] The token cache is being invalidate because project 0891d5f021664ff6b39f555c63ee5e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:59:59.460876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.461504 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6abba7c1-6221-41e9-87cb-15541747f271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:59:59.463019 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1398/2788] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 15:59:59 2022] DELETE /identity/v3/projects/0891d5f021664ff6b39f555c63ee5e5e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:59:59.468107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.468395 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.468672 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.490646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:59:59.492727 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:59.550217 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1391/2789] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:59:59.559128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.559413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.559689 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.580076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:59:59.581256 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:59.589067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1399/2790] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:59:59.593321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.602966 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa3aa6-8385-4e4c-aefd-c903c6e3046f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.607381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.607968 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.608240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.608506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.609140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:59:59.609440 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.611069 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.612472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:59:59.625043 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1392/2791] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 15:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:59:59.629069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.638862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eb8cc-8d8f-4dc4-ae01-a27ba0867888 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.643440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.644026 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.644303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.644572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.645201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.647101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.676856 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1400/2792] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 15:59:59 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:59:59.681613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.692077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194618-2656-45c2-931e-ca1df6822a61 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.747313 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.747907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.748182 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.748449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.749077 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.751039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.777178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1393/2793] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 15:59:59 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:59:59.781555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.792500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8044d-8a9b-4d07-a462-e21cdeacde99 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.797238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.797831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.798110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.798385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.799061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.800643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.811503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1401/2794] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:59:59.815541 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.825410 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711666-7999-4a2b-879c-82e1c0e81db0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.829892 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.830517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.830796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.831067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.831708 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.833315 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bdfa7-5e67-45cd-b35e-e331a430e130 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.843154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1394/2795] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 15:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:59:59.847381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.857853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbe50-93c2-4779-9a1f-0b4e2dbbe6c5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:59:59.862239 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:59:59.862964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.863216 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.863446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] PATH_INFO: `/v3/projects/9cf83fa74c6648ba959db6679857c466/users/ef337fd87e3a46c68339951fc72a281b/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.878184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf83fa74c6648ba959db6679857c466, user_id=ef337fd87e3a46c68339951fc72a281b, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:59:59.880928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:59:59.890041 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:59:59.895170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1402/2796] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 15:59:59 2022] PUT /identity/v3/projects/9cf83fa74c6648ba959db6679857c466/users/ef337fd87e3a46c68339951fc72a281b/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:59:59.900341 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee22b-3b9c-4519-b20a-41be9d6e603b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:59:59.900624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee22b-3b9c-4519-b20a-41be9d6e603b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:59:59.900895 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee22b-3b9c-4519-b20a-41be9d6e603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:59:59.921349 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-915ee22b-3b9c-4519-b20a-41be9d6e603b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:59:59.922563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-915ee22b-3b9c-4519-b20a-41be9d6e603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:59:59.979757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1395/2797] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 15:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:59:59.989631 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:59:59.991404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:00.048707 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5cf5-44be-4b31-b333-4b2fe625f34a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:00:00.052467 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:00:00.053069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:00.053351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:00.053625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:00.063533 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:00:00.064956 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:00:00.074372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1403/2798] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 15:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:00:03.950343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbbb89-ef7b-4876-800b-3de70c415b3a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:03.950912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbbb89-ef7b-4876-800b-3de70c415b3a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:03.950912 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbbb89-ef7b-4876-800b-3de70c415b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:03.979992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9dbbbb89-ef7b-4876-800b-3de70c415b3a None None] MFA Rules not processed for user `ef337fd87e3a46c68339951fc72a281b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:00:03.981147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9dbbbb89-ef7b-4876-800b-3de70c415b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:04.051726 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1396/2799] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:00:04.066877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:00:04.068331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:04.144897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792406a-9697-4b2f-bfe4-4d5d54880058 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:00:04.151104 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:00:04.151719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:04.152007 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:04.152286 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:04.161657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:00:04.164494 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:00:04.186877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1404/2800] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:00:05.537690 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:05.538216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:05.538216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:05.546122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:00:05.559704 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] MFA Rules not processed for user `ef337fd87e3a46c68339951fc72a281b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:00:05.560915 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:05.568098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1397/2801] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:00:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:00:06.221379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:00:06.222949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:06.313944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73848c3d-f325-47a9-932b-739cb2acd5b2 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:00:06.319032 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:00:06.319711 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:06.320129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:06.321004 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:06.322934 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:06.334493 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:00:06.336101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163aba91-6dd0-4979-a44a-3a8989a90086 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:00:06.346157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1405/2802] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:00:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:00:14.706865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:00:14.721423 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cfe4-9b47-4f49-b44c-b3f9213dd62b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:00:14.727034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:00:14.727646 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:14.727924 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:14.728195 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:14.729911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:14.742650 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:00:14.744216 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:00:14.754943 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1398/2803] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:00:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:00:16.342551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:00:24.750253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=8 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:01:33.199427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01991490-ce6e-465f-ab17-f00ceb1535fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:01:33.199955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01991490-ce6e-465f-ab17-f00ceb1535fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:01:33.199955 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01991490-ce6e-465f-ab17-f00ceb1535fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:01:33.230401 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-01991490-ce6e-465f-ab17-f00ceb1535fc None None] MFA Rules not processed for user `ef337fd87e3a46c68339951fc72a281b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:01:33.231645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-01991490-ce6e-465f-ab17-f00ceb1535fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:01:33.256998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1406/2804] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:01:43.255170 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=15 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:01:55.601885 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:01:55.659179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983ed4-b85f-45d7-baa6-b811f610ea3a service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:01:55.664168 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:01:55.664755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:01:55.665032 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:01:55.665301 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:01:55.667645 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:01:55.692011 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:01:55.693590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:01:55.703777 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1399/2805] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:02:05.699385 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=32 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:03:00.732619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:03:00.733224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:03:00.733224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:03:00.752569 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] MFA Rules not processed for user `ef337fd87e3a46c68339951fc72a281b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:03:00.753753 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:03:00.761002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1407/2806] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:03:04.913156 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:03:04.933094 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c818a5-8867-4786-864f-69ebaa5cb7bb service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:03:04.936660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:03:04.937253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:03:04.937799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:03:04.938071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:03:04.939826 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:03:04.971299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:03:04.972720 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a6f6e-4a21-4440-bcca-15cf4b43ea76 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:03:04.993110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1400/2807] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:03:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:03:10.747736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:03:14.990534 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=22 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:04:06.327352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:04:06.337153 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1437600-1347-4886-90df-7da23f0b7e3a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:04:06.340682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:04:06.341276 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:04:06.341551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:04:06.341824 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:04:06.343584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:04:06.366488 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:04:06.367905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:04:06.376480 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1408/2808] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:04:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:04:16.373356 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=11 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:04:42.396921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:04:42.397453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:04:42.397560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:04:42.416938 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] MFA Rules not processed for user `ef337fd87e3a46c68339951fc72a281b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:04:42.418147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:04:42.426389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1401/2809] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:04:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:04:42.734647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:04:42.743260 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9454b-636c-4f03-9acd-93a135156f7e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:04:42.746867 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:04:42.747465 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:04:42.747739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:04:42.748018 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:04:42.749854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:04:42.760370 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:04:42.761831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:04:42.770103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1409/2810] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:04:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:04:52.411347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:04:52.766597 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:04:53.863428 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:04:53.878138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03f67-f85f-4ff5-bca7-756a9e376279 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:04:53.885034 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:04:53.885595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:04:53.885874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:04:53.886147 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:04:53.887976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:04:53.900657 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:04:53.902242 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:04:53.912919 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1402/2811] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:04:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:05:03.908316 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:05:58.594542 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:05:58.608687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0209bf5-131f-46c5-aacf-475f7c4b0a10 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:05:58.614156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:05:58.614776 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:05:58.615055 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:05:58.615326 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:05:58.623123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:05:58.624687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:05:58.633584 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1410/2812] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:08.629854 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:17.456007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:17.465623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce04bf7-5954-4361-8c87-6243f3721f0b service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:17.469833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:17.470624 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:17.470950 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:17.471267 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:17.480744 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:17.482170 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:17.509843 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1403/2813] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:20.639754 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:20.641204 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:20.655931 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a913b13-c60d-4c9a-89bb-3c3bfe126aed service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:20.661058 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:20.661659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:20.661935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:20.662208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:20.664191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:20.681870 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:20.683683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:20.696150 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:20.702772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1411/2814] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:20.706373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b99ed-ddf1-4092-932e-b4bae4fc58ad service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:20.718332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:20.718542 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:20.718850 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:20.719122 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] PATH_INFO: `/v3/users/ef337fd87e3a46c68339951fc72a281b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:20.720321 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef337fd87e3a46c68339951fc72a281b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:20.722282 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:20.770106 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef337fd87e3a46c68339951fc72a281b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:20.776982 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef337fd87e3a46c68339951fc72a281b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:20.777579 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef337fd87e3a46c68339951fc72a281b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:20.778181 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef337fd87e3a46c68339951fc72a281b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:20.785757 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6173ad0a-c13a-4508-b21b-bd32f272644b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef337fd87e3a46c68339951fc72a281b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:20.807067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1404/2815] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:20 2022] DELETE /identity/v3/users/ef337fd87e3a46c68339951fc72a281b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:06:20.993694 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.004181 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf62b6-978b-4f6e-a240-1714fd398dae service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.184361 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.185012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.185287 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.185559 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] PATH_INFO: `/v3/projects/9cf83fa74c6648ba959db6679857c466` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.186686 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf83fa74c6648ba959db6679857c466)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.188815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.235951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf83fa74c6648ba959db6679857c466', 'request_id': 'req-0f927370-3057-4c46-a935-a13b26b9eedc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:21.241470 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf83fa74c6648ba959db6679857c466', 'request_id': 'req-0f927370-3057-4c46-a935-a13b26b9eedc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:21.248306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf83fa74c6648ba959db6679857c466', 'request_id': 'req-0f927370-3057-4c46-a935-a13b26b9eedc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:21.271970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] The token cache is being invalidate because project 9cf83fa74c6648ba959db6679857c466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:21.272258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:21.272873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0f927370-3057-4c46-a935-a13b26b9eedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:21.274340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1412/2816] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:20 2022] DELETE /identity/v3/projects/9cf83fa74c6648ba959db6679857c466 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:21.280194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.280194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.280462 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.338303 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:21.339544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:21.397371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1405/2817] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:21.405902 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.406178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.406473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.427323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:21.428500 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:21.436344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1413/2818] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:21.440756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.451740 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d7baa-6482-4eec-b369-6ae3ea5dc7a9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.456563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.457157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.457435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.457703 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.458344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 16:06:21.458674 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.460265 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.470086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:21.474815 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1406/2819] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 16:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 16:06:21.478048 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.487249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3eb636-21af-44e9-a0da-ba27d2bffbfa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.491993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.492598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.492869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.493139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.493765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.495739 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.538618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1414/2820] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 16:06:21.543005 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.553537 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bec19-8c44-4de5-b6b6-a5e8f7159fd8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.614188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.614818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.615097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.615366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.615988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.617888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.640074 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1407/2821] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 16:06:21.644244 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.654683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3db1bf-67ce-4962-b996-c087d878421e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.659154 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.659745 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.660017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.660282 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.660937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.662539 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.672268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1415/2822] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:21.676266 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.687679 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd838f-d824-463a-b209-7ff1d0695a05 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.692824 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.693422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.693699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.693971 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.694987 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.696592 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e04c9-70b8-4c03-975e-76ebcdbf8a82 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.715302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1408/2823] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:21.721687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.731880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ffdf-b3ac-4b6d-bc82-1c66d6efb1d0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.736261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.736894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.737172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.737437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] PATH_INFO: `/v3/projects/8318d64578044b7d98e8709d9649cde8/users/952b6160f2274613b9e9563f32b8f770/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.754964 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8318d64578044b7d98e8709d9649cde8, user_id=952b6160f2274613b9e9563f32b8f770, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.757577 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.759728 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:21.771799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1416/2824] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:21 2022] PUT /identity/v3/projects/8318d64578044b7d98e8709d9649cde8/users/952b6160f2274613b9e9563f32b8f770/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:06:21.781972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.782261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.782587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.809678 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] MFA Rules not processed for user `952b6160f2274613b9e9563f32b8f770`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:21.811436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:21.875296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1409/2825] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:21.880165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.942737 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1c38e-d0c9-4813-ae61-3f5ccbe28ae7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:21.947483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:21.948080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:21.948395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:21.948627 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:21.949259 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:21.951792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:21.978351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1417/2826] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:21.983097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:21.993709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf2dd4-527d-4416-8f67-babfd61c860e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.066719 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.067330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.067643 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.067911 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.068547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.070520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.086402 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:22.093397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1410/2827] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:21 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:22.097172 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.107882 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9184-509b-4444-8dd2-ac8331689f9f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.112555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.113142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.113422 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.113697 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.114352 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.115965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.126319 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1418/2828] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 16:06:22.131112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.141070 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891039f7-e2c7-4eeb-bd53-7d57fdee3821 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.145921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.146590 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.146871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.147146 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] PATH_INFO: `/v3/projects/585f9166952f47868826bb9a56840bec/users/794284ea5aa34cac927561d889ca3484/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.162474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585f9166952f47868826bb9a56840bec, user_id=794284ea5aa34cac927561d889ca3484, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.165136 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.180453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1411/2829] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:22 2022] PUT /identity/v3/projects/585f9166952f47868826bb9a56840bec/users/794284ea5aa34cac927561d889ca3484/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:06:22.184445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.195365 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212239-214f-4954-9489-705c9b5c076b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.251025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.251692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.251972 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.252243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.252885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.254486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38b88-8ac1-47fa-bd2c-4c552c151662 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.264240 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1419/2830] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:22.268503 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.278972 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cf585-d0ce-4e9a-a1d3-c1bbf07a1a41 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.283660 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.284302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.284583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.284860 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] PATH_INFO: `/v3/projects/585f9166952f47868826bb9a56840bec/users/794284ea5aa34cac927561d889ca3484/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.287089 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585f9166952f47868826bb9a56840bec, user_id=794284ea5aa34cac927561d889ca3484, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.289725 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.300344 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1412/2831] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:22 2022] PUT /identity/v3/projects/585f9166952f47868826bb9a56840bec/users/794284ea5aa34cac927561d889ca3484/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:22.312227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.312618 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.313192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.340587 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] MFA Rules not processed for user `794284ea5aa34cac927561d889ca3484`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:22.342314 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:22.391863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:22.413251 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1420/2832] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:22.424668 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.426371 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:22.509092 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c55e6-2883-491c-8764-f5cc47116039 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.512665 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.513273 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.513544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.513809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.522884 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.524307 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cb74e-e005-4d2b-be7f-623fbb1cd29f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.533979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1413/2833] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:22.667951 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.729219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f2fa4-5f43-49e6-a6a0-f4754a0c2fe7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.733652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.734267 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.734576 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.734849 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] PATH_INFO: `/v3/users/952b6160f2274613b9e9563f32b8f770` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.735949 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952b6160f2274613b9e9563f32b8f770)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.737831 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.782759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952b6160f2274613b9e9563f32b8f770'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:22.789374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952b6160f2274613b9e9563f32b8f770'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:22.790035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952b6160f2274613b9e9563f32b8f770'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:22.790652 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952b6160f2274613b9e9563f32b8f770'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:22.797862 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952b6160f2274613b9e9563f32b8f770'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:22.815727 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1421/2834] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:22 2022] DELETE /identity/v3/users/952b6160f2274613b9e9563f32b8f770 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:22.820245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.820747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.820863 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.843178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:22.844166 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:22.903023 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1414/2835] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:22.914012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:22.915442 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:22.971853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a783-582c-4b94-aa80-5f2dcd3c9352 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:22.975487 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:22.976086 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:22.976369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:22.976657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:22.985687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:22.987186 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:22.996581 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1422/2836] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:23.272020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:23.273762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:23.291713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f527f36-5630-454f-a36e-2ecc77f1fd55 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:23.296419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:23.297055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:23.297362 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:23.297636 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] PATH_INFO: `/v3/projects/8318d64578044b7d98e8709d9649cde8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:23.298789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8318d64578044b7d98e8709d9649cde8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:23.300827 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:23.341833 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:23.349921 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8318d64578044b7d98e8709d9649cde8', 'request_id': 'req-a4b95d5f-9aec-4232-9782-c965d12d26e0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.357054 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8318d64578044b7d98e8709d9649cde8', 'request_id': 'req-a4b95d5f-9aec-4232-9782-c965d12d26e0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.363688 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8318d64578044b7d98e8709d9649cde8', 'request_id': 'req-a4b95d5f-9aec-4232-9782-c965d12d26e0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.374685 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] The token cache is being invalidate because project 8318d64578044b7d98e8709d9649cde8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:23.374976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.375599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.377083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1415/2837] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:23 2022] DELETE /identity/v3/projects/8318d64578044b7d98e8709d9649cde8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:23.380933 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:23.390962 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c683f2-26f3-4621-8560-e3c271c41c67 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:23.398348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:23.454126 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:23.454784 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:23.455067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:23.455341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] PATH_INFO: `/v3/users/794284ea5aa34cac927561d889ca3484` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:23.456449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=794284ea5aa34cac927561d889ca3484)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:23.458340 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:23.505446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794284ea5aa34cac927561d889ca3484'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.512495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794284ea5aa34cac927561d889ca3484'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.513183 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794284ea5aa34cac927561d889ca3484'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.513878 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794284ea5aa34cac927561d889ca3484'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.521474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6b1ab215-557d-4fe4-babd-006c00a25115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794284ea5aa34cac927561d889ca3484'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.538322 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1423/2838] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:23 2022] DELETE /identity/v3/users/794284ea5aa34cac927561d889ca3484 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:23.827108 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:23.828557 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:23.841599 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95d5f-9aec-4232-9782-c965d12d26e0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:23.898897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:23.899521 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:23.899795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:23.900064 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] PATH_INFO: `/v3/projects/585f9166952f47868826bb9a56840bec` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:23.901188 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=585f9166952f47868826bb9a56840bec)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:23.903118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:23.947706 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585f9166952f47868826bb9a56840bec', 'request_id': 'req-c97849e9-3528-4b0f-90cc-22bee3398539'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.955634 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585f9166952f47868826bb9a56840bec', 'request_id': 'req-c97849e9-3528-4b0f-90cc-22bee3398539'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.962045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585f9166952f47868826bb9a56840bec', 'request_id': 'req-c97849e9-3528-4b0f-90cc-22bee3398539'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.972442 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] The token cache is being invalidate because project 585f9166952f47868826bb9a56840bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:23.972729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.973366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c97849e9-3528-4b0f-90cc-22bee3398539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:23.974874 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1416/2839] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:23 2022] DELETE /identity/v3/projects/585f9166952f47868826bb9a56840bec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:23.979798 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:23.980283 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:23.980396 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:23.999914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:24.001628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:24.074944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1424/2840] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 16:06:24.083353 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.083641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.083948 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.103617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:24.104806 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:24.113677 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1417/2841] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:24.117840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.127432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d40c7e-558b-4dc2-83ee-c2162d016958 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.131937 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.132531 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.132810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.133082 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.133725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 16:06:24.134029 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.135661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.147039 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1425/2842] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 16:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 16:06:24.151360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.160918 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e481640-708c-4586-9a80-1f27197cb30d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.165578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.166218 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.166524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.166805 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.167443 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.169907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.196792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1418/2843] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:24.201810 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.211563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa3a0-c598-4ffb-87dc-849af4dd1857 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.224101 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:24.270157 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.270797 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.271076 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.271347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.271971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.273893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.295227 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1426/2844] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:24.299343 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.309595 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89c51f-96e9-4f99-9a8f-2ad1a994cf28 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.315107 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.315695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.315973 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.316244 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.316882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.318511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.328691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1419/2845] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:24.332749 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.343065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dedd8-3676-428d-aa2a-9e5d159de660 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.352876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.353512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.353792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.354069 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.354744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.356344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef20c59-2572-443d-9fa6-dbd3244ef895 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.369751 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1427/2846] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:24.374695 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.385460 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92a0fa-8b6a-4781-afc7-83e21b5c5485 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.390332 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.391018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.391302 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.391603 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] PATH_INFO: `/v3/projects/c10f1709a18046c9a42922fc8f556dc6/users/9f717b7bf300428889773dad17865ecf/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.406865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c10f1709a18046c9a42922fc8f556dc6, user_id=9f717b7bf300428889773dad17865ecf, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.409449 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.411647 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:24.424593 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1420/2847] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:24 2022] PUT /identity/v3/projects/c10f1709a18046c9a42922fc8f556dc6/users/9f717b7bf300428889773dad17865ecf/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:24.434716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.435653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.435932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.462997 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] MFA Rules not processed for user `9f717b7bf300428889773dad17865ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:24.464208 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:24.527676 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1428/2848] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:24.532205 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.596737 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae286-23fd-4046-bfa3-b588e289d89a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.601445 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.602044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.602329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.602635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.603280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.605184 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.631838 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1421/2849] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:24.635954 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.646541 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40076c-d29c-4691-8862-a82a8ff8324d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.707247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.707759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.708035 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.708306 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.708928 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.711137 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.732863 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1429/2850] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:24.736906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.747154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa2bf-9676-4b83-8434-2069f099a155 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.751802 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.752392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.752675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.752951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.753600 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.755230 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.765796 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1422/2851] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:24.769706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.780119 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff631c2-fe73-422b-b11f-cd64768b2f74 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.784681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.785266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.785544 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.785813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.786473 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.788063 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d85ec9-c5be-4ea3-b872-8852ef4b0e4d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.798369 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1430/2852] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:24.802518 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:24.812451 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30345e23-88f7-4f04-b9ca-93dfc89ab13b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:24.817387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:24.818027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.818309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.818614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] PATH_INFO: `/v3/projects/c9006eb8b25045cf8616ab0127e17e63/users/77df356d4ea847d392af5c3c3b097ce7/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.833033 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9006eb8b25045cf8616ab0127e17e63, user_id=77df356d4ea847d392af5c3c3b097ce7, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:24.835723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:24.852598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1423/2853] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:24 2022] PUT /identity/v3/projects/c9006eb8b25045cf8616ab0127e17e63/users/77df356d4ea847d392af5c3c3b097ce7/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:24.864563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:24.864873 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:24.865147 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:24.873224 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:24.894625 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] MFA Rules not processed for user `77df356d4ea847d392af5c3c3b097ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:24.895815 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:24.961380 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1431/2854] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:24.965931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.028211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd861-9a80-49aa-be1c-802209c745c4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.033508 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.034105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.034387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.034691 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.035320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.037235 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.065219 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1424/2855] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:24 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:25.069565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.080372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36581-967f-4292-b94b-26c4535c430e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.136457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.137096 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.137344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.137621 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.138385 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.140508 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.160594 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1432/2856] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:25 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:25.165221 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.176659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d09855-ca43-43a9-8b49-16a65838a84b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.181408 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.182007 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.182292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.182601 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.183272 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.185473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.196271 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1425/2857] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:25.200377 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.210061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e59c4-179c-4d93-b699-e5c7ae6afc2c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.214696 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.215284 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.215565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.215840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] PATH_INFO: `/v3/projects/d6c6c90670974c9b8e387484f4fc7b56/users/914110606c15469d8771d08fe44659dc/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.230443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c6c90670974c9b8e387484f4fc7b56, user_id=914110606c15469d8771d08fe44659dc, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.233630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.246468 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1433/2858] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:25 2022] PUT /identity/v3/projects/d6c6c90670974c9b8e387484f4fc7b56/users/914110606c15469d8771d08fe44659dc/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:25.251189 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.261480 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976548b0-b4e4-4e5a-bebf-a64cb4e5c5bd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.274598 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:25.320747 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.321354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.321637 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.321929 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.322587 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.324175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf902e8-3d97-42c4-9a22-e07da34c1a56 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.333880 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1426/2859] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:25.338042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.348570 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603048-137b-41d0-9214-b222d25c880c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.353348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.353994 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.354274 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.354578 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] PATH_INFO: `/v3/projects/d6c6c90670974c9b8e387484f4fc7b56/users/914110606c15469d8771d08fe44659dc/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.356512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c6c90670974c9b8e387484f4fc7b56, user_id=914110606c15469d8771d08fe44659dc, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.359713 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.369843 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1434/2860] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:25 2022] PUT /identity/v3/projects/d6c6c90670974c9b8e387484f4fc7b56/users/914110606c15469d8771d08fe44659dc/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:06:25.380253 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.380544 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.380818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.408800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] MFA Rules not processed for user `914110606c15469d8771d08fe44659dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:25.410013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:25.476356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1427/2861] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:25.489044 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.490782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:25.541880 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:25.549120 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5a3e-846c-4b87-b72c-62f5f3a76e42 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.552965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.553572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.553847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.554233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.563376 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.564822 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.574083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1435/2862] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:25.617140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:25.618621 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:25.630563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027745a-5828-4fef-9249-6a311564d5fd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:25.634373 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:25.635328 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:25.635614 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:25.635890 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:25.694774 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:25.696818 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8d890-6b6a-40db-9a39-239043a25356 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:25.707299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1428/2863] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:28.586337 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:28.646549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7193-34bc-4d16-beb0-ca33bd2b61a8 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:28.651129 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:28.651748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:28.652022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:28.652295 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] PATH_INFO: `/v3/users/9f717b7bf300428889773dad17865ecf` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:28.653869 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f717b7bf300428889773dad17865ecf)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:28.655790 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:28.700097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f717b7bf300428889773dad17865ecf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:28.707431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f717b7bf300428889773dad17865ecf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:28.708077 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f717b7bf300428889773dad17865ecf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:28.708669 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f717b7bf300428889773dad17865ecf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:28.717272 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f717b7bf300428889773dad17865ecf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:28.735099 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1436/2864] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:28 2022] DELETE /identity/v3/users/9f717b7bf300428889773dad17865ecf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:06:28.740269 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:28.740847 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:28.741417 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:28.762130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:28.763419 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:28.795494 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:28.823237 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1429/2865] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:28.833014 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:28.834404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:28.898146 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98b81-6bd2-4180-9922-96c4f584de54 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:28.901726 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:28.902323 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:28.902624 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:28.902894 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:28.911659 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:28.913066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:28.922847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1437/2866] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:29.193236 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:29.194931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:29.207333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9b21a-ac78-4d91-95f2-719ed83ff941 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:29.212935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:29.213583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:29.213866 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:29.214143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] PATH_INFO: `/v3/projects/c10f1709a18046c9a42922fc8f556dc6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:29.215339 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c10f1709a18046c9a42922fc8f556dc6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:29.217830 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:29.266823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c10f1709a18046c9a42922fc8f556dc6', 'request_id': 'req-8d979d08-a064-46f8-8070-9ccd9a70229f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.274659 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c10f1709a18046c9a42922fc8f556dc6', 'request_id': 'req-8d979d08-a064-46f8-8070-9ccd9a70229f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.281666 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c10f1709a18046c9a42922fc8f556dc6', 'request_id': 'req-8d979d08-a064-46f8-8070-9ccd9a70229f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.291967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] The token cache is being invalidate because project c10f1709a18046c9a42922fc8f556dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:29.292261 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.292906 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.295062 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1430/2867] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:29 2022] DELETE /identity/v3/projects/c10f1709a18046c9a42922fc8f556dc6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:29.299168 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:29.308690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3b17-3f1f-4854-aa42-bc931d043a92 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:29.316746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:29.366695 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:29.367328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:29.367610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:29.367885 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] PATH_INFO: `/v3/users/77df356d4ea847d392af5c3c3b097ce7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:29.368989 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77df356d4ea847d392af5c3c3b097ce7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:29.370970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:29.415723 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77df356d4ea847d392af5c3c3b097ce7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.422990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77df356d4ea847d392af5c3c3b097ce7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.423646 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77df356d4ea847d392af5c3c3b097ce7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.424253 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77df356d4ea847d392af5c3c3b097ce7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.431414 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77df356d4ea847d392af5c3c3b097ce7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.448549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1438/2868] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:29 2022] DELETE /identity/v3/users/77df356d4ea847d392af5c3c3b097ce7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:29.722240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:29.723756 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:29.738192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d979d08-a064-46f8-8070-9ccd9a70229f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:29.798366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:29.799099 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:29.799280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:29.799551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] PATH_INFO: `/v3/projects/c9006eb8b25045cf8616ab0127e17e63` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:29.800642 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9006eb8b25045cf8616ab0127e17e63)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:29.802550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:29.848487 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9006eb8b25045cf8616ab0127e17e63', 'request_id': 'req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.856078 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9006eb8b25045cf8616ab0127e17e63', 'request_id': 'req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.862560 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9006eb8b25045cf8616ab0127e17e63', 'request_id': 'req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.864552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:29.875583 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] The token cache is being invalidate because project c9006eb8b25045cf8616ab0127e17e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:29.876192 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.876869 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:29.878382 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1431/2869] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:29 2022] DELETE /identity/v3/projects/c9006eb8b25045cf8616ab0127e17e63 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:29.882529 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:29.884219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:29.897339 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551b7d-8a9a-446e-8d81-caf8a2fca0f6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:29.953503 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:29.954128 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:29.954425 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:29.954698 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] PATH_INFO: `/v3/users/914110606c15469d8771d08fe44659dc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:29.955796 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914110606c15469d8771d08fe44659dc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:29.957680 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:29.990065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:30.003886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914110606c15469d8771d08fe44659dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.010744 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914110606c15469d8771d08fe44659dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.011391 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914110606c15469d8771d08fe44659dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.011979 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914110606c15469d8771d08fe44659dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.019978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-565f86de-0f52-4806-9516-9065d1cf49f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914110606c15469d8771d08fe44659dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.037403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1439/2870] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:29 2022] DELETE /identity/v3/users/914110606c15469d8771d08fe44659dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:30.317278 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.318742 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:30.333612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77cfe9-9cc6-4fe1-94cb-27a61990c5f0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.392044 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.392619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.392894 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.393167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] PATH_INFO: `/v3/projects/d6c6c90670974c9b8e387484f4fc7b56` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.394259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c6c90670974c9b8e387484f4fc7b56)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.396193 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.442280 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c6c90670974c9b8e387484f4fc7b56', 'request_id': 'req-cfd7de67-c785-4360-ae13-4a2ca8d08e61'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.449486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c6c90670974c9b8e387484f4fc7b56', 'request_id': 'req-cfd7de67-c785-4360-ae13-4a2ca8d08e61'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.456140 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c6c90670974c9b8e387484f4fc7b56', 'request_id': 'req-cfd7de67-c785-4360-ae13-4a2ca8d08e61'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.466264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] The token cache is being invalidate because project d6c6c90670974c9b8e387484f4fc7b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:30.466582 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.467225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfd7de67-c785-4360-ae13-4a2ca8d08e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:30.468700 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1432/2871] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:30 2022] DELETE /identity/v3/projects/d6c6c90670974c9b8e387484f4fc7b56 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:30.475271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.475557 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.475830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.499524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:30.500706 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:30.557791 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1440/2872] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:30.568475 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.568765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.569041 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.590566 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:30.592285 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:30.600257 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1433/2873] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:30.604547 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.614803 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026bcd5-5c8d-4048-8a4e-d57ea5a35484 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.619325 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.619919 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.620194 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.620463 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.621097 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 16:06:30.621404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.623124 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.634393 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1441/2874] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 16:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 16:06:30.638956 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.648183 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784bd2b-2fb9-4ed8-afec-533a281cfc89 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.652867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.653951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.654226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.654524 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.655167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.657047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.688101 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1434/2875] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:30 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:30.692271 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.702123 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa942-481e-423b-b935-c071a4c1c124 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.758764 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.759367 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.759643 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.759914 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.760546 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.762456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.788379 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1442/2876] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:30 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:30.796143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.804177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad59a6a-d9b7-4692-a183-cabcdff98351 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.808867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.809492 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.809823 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.810098 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.810765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.812354 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.823352 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1435/2877] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:30.828675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.839524 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f73-0962-486f-85d2-020a52a3acca admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.845257 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.845847 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.846127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.846395 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.847062 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.848716 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b5df6-5c33-4102-9803-430383a8200d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.861456 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1443/2878] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:30.865926 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:30.878486 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc22200-ff90-4a34-aa45-13f6f9bac6d3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:30.883366 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:30.884049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.884331 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.884609 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] PATH_INFO: `/v3/projects/dcd28f150ad5435a94e47f02c967e213/users/6046df1f4c8f4e9faeb93634d8a52505/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.902112 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd28f150ad5435a94e47f02c967e213, user_id=6046df1f4c8f4e9faeb93634d8a52505, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:30.905259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:30.918109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1436/2879] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:30 2022] PUT /identity/v3/projects/dcd28f150ad5435a94e47f02c967e213/users/6046df1f4c8f4e9faeb93634d8a52505/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:30.929601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:30.930113 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:30.930221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:30.962892 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] MFA Rules not processed for user `6046df1f4c8f4e9faeb93634d8a52505`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:30.964122 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:30.969252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:31.030922 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1444/2880] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:31.035409 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.053939 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:31.098611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f17e-f137-4bab-bba5-03bb71915db3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.103246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.103848 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.104128 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.104399 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.105020 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.106951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.145531 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1437/2881] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:31.149692 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.161296 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45d295-4bc5-4793-8e43-039fb2787e40 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.219944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.220563 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.220895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.221190 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.221775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.223708 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.245213 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1445/2882] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:31.249309 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.259617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f1f79-fd4f-4f1d-9b50-5028d1fe3b9a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.264795 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.265638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.265638 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.265893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.266564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.268163 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.278867 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1438/2883] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:31.283853 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.296317 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692fbb-1554-4bd6-ac32-d2792b8f8971 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.301017 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.301614 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.301897 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.302166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.302851 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.304426 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d12441-be66-45e5-b7f2-4bdc7808148d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.315333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1446/2884] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 16:06:31.319569 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.330565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec90e-812a-4c5e-a1d6-e4eb14fa248c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.336138 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.336803 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.337113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.337392 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] PATH_INFO: `/v3/projects/52987498727947ac887fec382ed8fb85/users/84683a15ea154db19455800246f3f0e5/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.354175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52987498727947ac887fec382ed8fb85, user_id=84683a15ea154db19455800246f3f0e5, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.358270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.371899 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1439/2885] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:31 2022] PUT /identity/v3/projects/52987498727947ac887fec382ed8fb85/users/84683a15ea154db19455800246f3f0e5/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:31.383100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.383490 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.383794 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.411647 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] MFA Rules not processed for user `84683a15ea154db19455800246f3f0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:31.412786 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:31.475895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1447/2886] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:31.480105 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.540199 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de07b87-8cf4-42ca-a79c-acc73de7f490 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.544865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.545457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.545743 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.546024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.546709 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.548612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.575322 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1440/2887] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:31.580262 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.590768 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5abeb-3e81-49a4-ac55-a0e83f22c631 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.599328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:31.646564 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.647165 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.647445 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.647719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.648341 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.650252 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.671457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1448/2888] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 16:06:31.675389 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.685656 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1d427-7981-4d83-9a49-e935dddced07 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.691082 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.691705 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.692059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.692422 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.693148 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.694967 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.705436 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1441/2889] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:31.709719 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.719610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e0f64-6951-4f46-9a37-9a87c3f4393f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.724221 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.724861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.725142 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.725417 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] PATH_INFO: `/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be/users/92e1066845ac4ed084966d3bcf98cee6/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.741294 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1995d85e214eebbd2d4cb35c11e4be, user_id=92e1066845ac4ed084966d3bcf98cee6, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.743944 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.758243 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1449/2890] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:31 2022] PUT /identity/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be/users/92e1066845ac4ed084966d3bcf98cee6/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:31.762165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.772840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601a01e-9467-419c-8c51-0860acc66537 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.812059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:31.829764 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.830360 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.830680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.830935 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.831565 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.833143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7336af0-1aa6-409a-876f-9419fcff8a9e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.843204 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1442/2891] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:06:31.847420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:31.857709 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08d4f0-0d81-4bf1-bcf0-5f285872ebe9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:31.862895 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:31.863572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.863848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.864118 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] PATH_INFO: `/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be/users/92e1066845ac4ed084966d3bcf98cee6/roles/161e292b353449f4bc2d222d2312b3c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.865993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1995d85e214eebbd2d4cb35c11e4be, user_id=92e1066845ac4ed084966d3bcf98cee6, role_id=161e292b353449f4bc2d222d2312b3c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:31.868628 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:31.879042 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1450/2892] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:31 2022] PUT /identity/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be/users/92e1066845ac4ed084966d3bcf98cee6/roles/161e292b353449f4bc2d222d2312b3c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:31.891088 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:31.891378 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:31.891654 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:31.919625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] MFA Rules not processed for user `92e1066845ac4ed084966d3bcf98cee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:31.920882 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:31.987898 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1443/2893] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:31.999355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:32.001114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:32.057510 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad88d-722c-40ad-84db-d4fa5b7cd77e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:32.061583 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:32.062178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:32.062491 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:32.062765 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:32.117347 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:32.118813 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:32.127971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1451/2894] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:33.633851 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:33.635292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:33.647211 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c909f3b-eff4-4ef3-a3b4-d537c66391aa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:33.651143 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:33.651739 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:33.652018 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:33.652290 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:33.712471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:33.713984 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:33.723338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1444/2895] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:34.268513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:34.277223 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956b112-6813-4a2f-8928-d138fff00dc9 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:34.281366 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:34.281948 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:34.282220 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:34.282518 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:34.291249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:34.292638 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db2d9-2fcc-47a7-831b-4ca648ec5024 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:34.301861 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1452/2896] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:38.216792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:38.277840 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ac36-83fa-43ba-b368-f9db588c7a71 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:38.282292 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:38.282955 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:38.283249 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:38.283507 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] PATH_INFO: `/v3/users/6046df1f4c8f4e9faeb93634d8a52505` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:38.284617 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6046df1f4c8f4e9faeb93634d8a52505)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:38.286548 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:38.288724 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:38.332888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6046df1f4c8f4e9faeb93634d8a52505'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.339653 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6046df1f4c8f4e9faeb93634d8a52505'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.340299 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6046df1f4c8f4e9faeb93634d8a52505'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.340888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6046df1f4c8f4e9faeb93634d8a52505'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.348723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6046df1f4c8f4e9faeb93634d8a52505'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.366169 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1445/2897] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:38 2022] DELETE /identity/v3/users/6046df1f4c8f4e9faeb93634d8a52505 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:38.370877 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:38.371381 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:38.371507 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:38.385100 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:38.392248 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:38.393585 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:38.451612 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1453/2898] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:38.462578 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:38.462976 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:38.521133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c643-fa8a-48f2-ac46-5584e2ba7ce6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:38.524733 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:38.525333 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:38.525611 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:38.525897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:38.535087 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:38.536509 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:38.546319 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1446/2899] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:38.819619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:38.821040 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:38.837311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8666a-dc82-4b53-8420-fad48ebb8f9b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:38.843030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:38.843690 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:38.843971 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:38.844250 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] PATH_INFO: `/v3/projects/dcd28f150ad5435a94e47f02c967e213` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:38.847344 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd28f150ad5435a94e47f02c967e213)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:38.849404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:38.903598 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd28f150ad5435a94e47f02c967e213', 'request_id': 'req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.909650 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd28f150ad5435a94e47f02c967e213', 'request_id': 'req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.917095 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd28f150ad5435a94e47f02c967e213', 'request_id': 'req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.929780 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] The token cache is being invalidate because project dcd28f150ad5435a94e47f02c967e213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:38.930065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.930720 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:38.932191 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1454/2900] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:38 2022] DELETE /identity/v3/projects/dcd28f150ad5435a94e47f02c967e213 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:38.937561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:38.947506 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9b241-5913-4b8e-85c8-d7b0da66a13a service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:39.006387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:39.007039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:39.007330 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:39.007696 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] PATH_INFO: `/v3/users/84683a15ea154db19455800246f3f0e5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:39.008979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84683a15ea154db19455800246f3f0e5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:39.011012 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:39.038800 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:39.062084 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84683a15ea154db19455800246f3f0e5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.069231 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84683a15ea154db19455800246f3f0e5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.069888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84683a15ea154db19455800246f3f0e5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.070550 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84683a15ea154db19455800246f3f0e5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.077908 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84683a15ea154db19455800246f3f0e5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.096142 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1447/2901] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:38 2022] DELETE /identity/v3/users/84683a15ea154db19455800246f3f0e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:39.381917 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:39.383413 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:39.397348 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b529-4ad9-4e1b-8e8f-feefe8df0d1f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:39.462736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:39.463374 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:39.463658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:39.463935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] PATH_INFO: `/v3/projects/52987498727947ac887fec382ed8fb85` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:39.465269 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52987498727947ac887fec382ed8fb85)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:39.468906 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:39.509012 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:39.522551 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52987498727947ac887fec382ed8fb85', 'request_id': 'req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.527987 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52987498727947ac887fec382ed8fb85', 'request_id': 'req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.536775 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52987498727947ac887fec382ed8fb85', 'request_id': 'req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.548010 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] The token cache is being invalidate because project 52987498727947ac887fec382ed8fb85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:39.548328 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.548958 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.550899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1455/2902] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:39 2022] DELETE /identity/v3/projects/52987498727947ac887fec382ed8fb85 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:39.554776 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:39.556206 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:39.570039 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9195-7428-4129-ba97-86ff6efb4f25 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:39.625177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:39.625792 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:39.626075 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:39.626346 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] PATH_INFO: `/v3/users/92e1066845ac4ed084966d3bcf98cee6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:39.627472 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e1066845ac4ed084966d3bcf98cee6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:39.629384 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:39.675699 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e1066845ac4ed084966d3bcf98cee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.682517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e1066845ac4ed084966d3bcf98cee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.683179 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e1066845ac4ed084966d3bcf98cee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.683788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e1066845ac4ed084966d3bcf98cee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.691563 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e9dc4935-7d43-43ce-bd9c-17f15f5c173d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e1066845ac4ed084966d3bcf98cee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:39.710834 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1448/2903] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:06:39 2022] DELETE /identity/v3/users/92e1066845ac4ed084966d3bcf98cee6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:39.990565 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:39.992161 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:40.005656 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5875-7d6f-4ad7-a623-53cbda00f4c0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.064431 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.065065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.065345 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.065675 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] PATH_INFO: `/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.066774 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1995d85e214eebbd2d4cb35c11e4be)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.068681 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.115512 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1995d85e214eebbd2d4cb35c11e4be', 'request_id': 'req-d8de270d-455b-4c2e-9c07-0d977734891d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:40.121457 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1995d85e214eebbd2d4cb35c11e4be', 'request_id': 'req-d8de270d-455b-4c2e-9c07-0d977734891d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:40.129434 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1995d85e214eebbd2d4cb35c11e4be', 'request_id': 'req-d8de270d-455b-4c2e-9c07-0d977734891d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:40.140608 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] The token cache is being invalidate because project ca1995d85e214eebbd2d4cb35c11e4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:06:40.140891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:40.141519 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8de270d-455b-4c2e-9c07-0d977734891d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:06:40.143008 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1456/2904] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:06:39 2022] DELETE /identity/v3/projects/ca1995d85e214eebbd2d4cb35c11e4be => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:40.147797 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.148083 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.148356 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.168061 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:40.169250 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:40.187045 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:40.231755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1449/2905] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:40.240192 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.240483 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.240759 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.260936 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:40.262139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:40.270893 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1457/2906] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:40.275367 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.285543 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8f7b-39cc-44f0-a472-f98eeabdacf2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.289996 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.290613 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.290891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.291158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.291794 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 16:06:40.292093 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.293675 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.305547 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1450/2907] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 16:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 16:06:40.309986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.319601 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085e7d7-09bc-4cb2-b434-a4fb4a7682fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.324156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.324746 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.325027 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.325303 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.325973 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.327891 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.355750 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1458/2908] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:06:40 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:40.360491 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.370288 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0d308-a756-4230-b4d6-d99ead94d294 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.429275 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.429888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.430172 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.430474 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.431110 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.433056 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.454931 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1451/2909] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:06:40 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:06:40.459127 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.470351 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7926-4aac-403e-a7c6-1498bb1e67bf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.475178 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.475783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.476247 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.476523 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.477159 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.478792 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.491219 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1459/2910] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 16:06:40.495488 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.505514 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da96634-45a9-46e0-913b-930d24138132 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.510226 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.510852 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.511132 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.511404 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.512071 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.513680 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d58be-137d-4a8b-a0d7-640b08bab773 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.523432 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1452/2911] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 16:06:40.527553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.537829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6315-a251-4648-8444-3c6c76f43a39 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.542312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.542986 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.543266 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.543536 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] PATH_INFO: `/v3/projects/c5edadda817042b1af27039d30e8fa68/users/21d2ed0f86f141d299e5440bc54eb31b/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.552550 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:40.559534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5edadda817042b1af27039d30e8fa68, user_id=21d2ed0f86f141d299e5440bc54eb31b, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.563155 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.576683 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1460/2912] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:06:40 2022] PUT /identity/v3/projects/c5edadda817042b1af27039d30e8fa68/users/21d2ed0f86f141d299e5440bc54eb31b/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:06:40.586927 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88bf0-40b7-41e9-bcdf-780519ce709b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.587564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88bf0-40b7-41e9-bcdf-780519ce709b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.587564 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88bf0-40b7-41e9-bcdf-780519ce709b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.620576 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-88f88bf0-40b7-41e9-bcdf-780519ce709b None None] MFA Rules not processed for user `21d2ed0f86f141d299e5440bc54eb31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:40.621784 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-88f88bf0-40b7-41e9-bcdf-780519ce709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:40.688865 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1453/2913] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:40.699420 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:40.701626 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:40.757682 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1490d1-e6c3-4537-a6f1-dfb42919d535 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:40.761449 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:40.762064 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:40.762333 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:40.762634 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:40.771610 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:40.773022 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:40.782758 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1461/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:42.157001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:42.157555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:42.157555 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:42.176765 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] MFA Rules not processed for user `21d2ed0f86f141d299e5440bc54eb31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:42.178013 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:42.185175 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1454/2915] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:06:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:42.655372 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:42.656811 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:42.727474 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72cb68-fc41-4306-a6ae-088e51645d5a service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:42.732837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:42.733435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:42.733736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:42.734002 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:42.735724 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:42.746782 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:42.748355 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:42.758080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1462/2916] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:52.171974 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:52.753876 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:06:52.772891 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:52.774406 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:52.850178 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c779d-2bd1-4f60-b9d2-b75565a25e6b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:52.855686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fcc9eec4e114f538c506178c744b49a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:52.856502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:52.856502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:52.856873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:52.858712 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:52.870788 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:52.872574 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6eb3-c5d6-42be-af6f-1fa83c5a6729 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:52.883716 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1455/2917] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:58.298999 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:58.310527 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404beefb-5c05-4365-a891-34ac1143d092 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:58.315788 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:58.316403 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:58.316701 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:58.316950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:58.318679 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:58.330725 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:58.332300 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:58.342840 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1463/2918] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:06:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:06:59.441951 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d5b3-e087-4217-bb5c-8e939578f06d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:59.442457 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d5b3-e087-4217-bb5c-8e939578f06d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:59.442641 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d5b3-e087-4217-bb5c-8e939578f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:59.505888 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4512d5b3-e087-4217-bb5c-8e939578f06d None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:06:59.507145 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4512d5b3-e087-4217-bb5c-8e939578f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:59.570151 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1456/2919] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:06:59.616036 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:06:59.617446 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:06:59.705574 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bd64-2bf0-4886-b436-04cc338932a3 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:06:59.710572 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:06:59.711162 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:06:59.711435 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:06:59.711704 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:06:59.719651 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:06:59.721202 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:06:59.728889 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1464/2920] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 16:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 16:07:00.236729 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c9d1e-887e-4ff5-a978-a466f0a451f2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:00.237097 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c9d1e-887e-4ff5-a978-a466f0a451f2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:00.237375 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c9d1e-887e-4ff5-a978-a466f0a451f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:00.256588 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d37c9d1e-887e-4ff5-a978-a466f0a451f2 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:00.257786 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d37c9d1e-887e-4ff5-a978-a466f0a451f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:00.265901 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1457/2921] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:00.278258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:00.289268 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67f99-ed3d-404e-bcca-8d9e1ee1afd0 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:00.294270 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:00.294886 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:00.295166 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:00.295440 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:00.303378 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:00.304939 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:00.313141 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1465/2922] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 16:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 16:07:00.539245 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c1cc-4097-4772-a58c-a2cd30e6efbc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:00.539893 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c1cc-4097-4772-a58c-a2cd30e6efbc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:00.540177 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c1cc-4097-4772-a58c-a2cd30e6efbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:00.560118 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cea7c1cc-4097-4772-a58c-a2cd30e6efbc None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:00.561327 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cea7c1cc-4097-4772-a58c-a2cd30e6efbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:00.568905 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1458/2923] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:00.581630 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:00.592264 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca60b6-0335-4617-9379-0600d9bf63a3 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:00.597373 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:00.597905 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:00.598184 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:00.598549 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:00.606555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:00.608110 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:00.616067 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1466/2924] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 16:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 16:07:00.952979 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:00.953167 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:00.953397 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:00.975165 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] MFA Rules not processed for user `d5846f447c204206bde8acaa4160664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:00.976760 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:00.984873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1459/2925] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:00.997111 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:01.008311 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551153-590d-4517-9d78-764c7bd019c6 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:01.013899 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf1537e6fa46238223fd03ced27a43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:01.014534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:01.014812 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:01.015083 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:01.024028 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:01.025592 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a931b-55af-45c2-9e90-c02f5b08c3ef service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:01.034249 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1467/2926] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 16:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 16:07:03.329109 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:03.359686 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:03.391259 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0318b8b-c20d-4a0e-9bf5-1b74baa5d2d8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:03.395723 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:03.396347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:03.396625 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:03.396897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] PATH_INFO: `/v3/users/21d2ed0f86f141d299e5440bc54eb31b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:03.398125 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d2ed0f86f141d299e5440bc54eb31b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:03.400086 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:03.446055 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d2ed0f86f141d299e5440bc54eb31b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.453386 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d2ed0f86f141d299e5440bc54eb31b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.454040 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d2ed0f86f141d299e5440bc54eb31b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.454934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d2ed0f86f141d299e5440bc54eb31b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.461748 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d2ed0f86f141d299e5440bc54eb31b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.480024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1460/2927] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:07:03 2022] DELETE /identity/v3/users/21d2ed0f86f141d299e5440bc54eb31b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:07:03.484582 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:03.485065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:03.485173 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:03.507661 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:03.509571 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:03.570071 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1468/2928] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:03.581274 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:03.582721 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:03.641471 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d55a91-4023-49dd-aae1-ac78c038d394 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:03.645113 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:03.645713 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:03.645992 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:03.646264 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:03.655552 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:03.657024 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6f331-195a-4957-821c-cc035ebd76af service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:03.666663 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1461/2929] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 16:07:03.832970 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:03.834783 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:03.847543 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a7ddb-97b8-42e8-a5c4-4332dc8421ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:03.852429 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:03.853066 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:03.853346 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:03.853619 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] PATH_INFO: `/v3/projects/c5edadda817042b1af27039d30e8fa68` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:03.854762 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5edadda817042b1af27039d30e8fa68)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:03.857265 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:03.869748 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:03.905427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5edadda817042b1af27039d30e8fa68', 'request_id': 'req-38f234a7-1166-423d-b720-fae4e9a19ef4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.910881 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5edadda817042b1af27039d30e8fa68', 'request_id': 'req-38f234a7-1166-423d-b720-fae4e9a19ef4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.919829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5edadda817042b1af27039d30e8fa68', 'request_id': 'req-38f234a7-1166-423d-b720-fae4e9a19ef4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.930911 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] The token cache is being invalidate because project c5edadda817042b1af27039d30e8fa68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:07:03.931201 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.931837 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38f234a7-1166-423d-b720-fae4e9a19ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:07:03.933304 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1469/2930] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:07:03 2022] DELETE /identity/v3/projects/c5edadda817042b1af27039d30e8fa68 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:07:03.938517 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:03.938799 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:03.939067 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:03.958957 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:03.960208 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:04.004133 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:04.019878 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1462/2931] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:04.028941 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.029229 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.029502 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.053846 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:04.055167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:04.063485 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1470/2932] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:04.067809 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.078001 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494e6bd-0af4-4b57-b675-35d81a1185aa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.082837 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.083520 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.083710 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.084029 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.084934 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 16:07:04.085239 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.087157 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.100430 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1463/2933] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 16:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 16:07:04.105331 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.115555 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823de47-1f9c-4b28-8c61-484c8e8a900e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.120233 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.120829 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.121112 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.121387 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.122033 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.123959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.151908 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1471/2934] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 16:07:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:07:04.156864 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.166810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2da4-2974-4b9f-98b6-2caf835e5c78 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.223619 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.224225 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.224502 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.224775 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.225441 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.227394 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.248589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1464/2935] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 16:07:04 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 16:07:04.252827 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.263663 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c8e8-9f95-49fc-bbc2-0d930df8addf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.268179 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.268772 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.269045 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.269312 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.269998 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.271553 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.281978 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1472/2936] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:07:04.285988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.295897 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d24fc-46a0-4eb9-bdb8-4c37d6705d6c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.300453 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.301047 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.301325 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.301597 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.302240 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.304014 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee582b7-1fc2-47b3-848f-4209166a8d2d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.315581 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1465/2937] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 16:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 16:07:04.319802 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.330065 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dc7eb-5fc9-4f5f-a013-10cb956840e3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.334534 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.335167 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.335443 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.335710 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] PATH_INFO: `/v3/projects/9c2422a69a09434a80242c7572ba9e13/users/616d569f5bc24fb8bc5c78128ca4772f/roles/5a6068f0cdfb4778a8837b31039dbeb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.350121 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2422a69a09434a80242c7572ba9e13, user_id=616d569f5bc24fb8bc5c78128ca4772f, role_id=5a6068f0cdfb4778a8837b31039dbeb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.352838 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.366176 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1473/2938] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 16:07:04 2022] PUT /identity/v3/projects/9c2422a69a09434a80242c7572ba9e13/users/616d569f5bc24fb8bc5c78128ca4772f/roles/5a6068f0cdfb4778a8837b31039dbeb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:07:04.371194 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daeca-72ee-44f9-a21f-a2c0e503d61a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.371683 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daeca-72ee-44f9-a21f-a2c0e503d61a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.371789 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daeca-72ee-44f9-a21f-a2c0e503d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.392513 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f66daeca-72ee-44f9-a21f-a2c0e503d61a None None] MFA Rules not processed for user `a86c62b56d134029a7eefd7d39742f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:04.393714 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f66daeca-72ee-44f9-a21f-a2c0e503d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:04.451900 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1466/2939] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:04.461437 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:04.463114 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:04.519718 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9decf4-b5c2-4d55-85e4-0d0ba2dbd26a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:04.523455 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:04.523983 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:04.524261 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:04.524532 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:04.533354 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:04.534814 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:04.543959 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1474/2940] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:07:05.400612 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d387-3d7f-4b18-8e71-9e49bddfc50a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:05.401238 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d387-3d7f-4b18-8e71-9e49bddfc50a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:05.401551 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d387-3d7f-4b18-8e71-9e49bddfc50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:05.445435 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-add4d387-3d7f-4b18-8e71-9e49bddfc50a None None] MFA Rules not processed for user `616d569f5bc24fb8bc5c78128ca4772f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:05.446689 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-add4d387-3d7f-4b18-8e71-9e49bddfc50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:05.494059 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:05.516347 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1467/2941] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:05.527604 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:07:05.529152 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:05.574992 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:05.590258 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d8ab-cfb2-4a3c-a2fa-e9d957f115a9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:07:05.593982 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:07:05.594554 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:05.594832 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:05.595102 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:05.604068 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:07:05.605486 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:07:05.614871 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1475/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:07:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:07:06.984810 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:07:06.985345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:07:06.985345 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:07:07.004130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] MFA Rules not processed for user `616d569f5bc24fb8bc5c78128ca4772f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:07:07.005329 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:07:07.012154 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1468/2943] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:07:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:07:15.610868 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:07:16.999359 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=9 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:08:00.584613 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:08:00.594513 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e46-e9dc-4ea7-8110-5391caef4161 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:08:00.598025 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:08:00.598645 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:00.598926 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:00.599203 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:00.600969 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:00.671658 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:08:00.673080 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a945c3d-0f76-4e56-bd49-a71644012858 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:08:00.684830 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1476/2944] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:08:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:08:00.692875 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:08:00.702787 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbac8-cf27-4dcc-b621-53f5aa6d2355 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:08:00.706447 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:08:00.707057 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:00.707338 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:00.707635 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:00.709505 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] Could not find project: bcffff710dd74209b30a1d02fb2bec46.: keystone.exception.ProjectNotFound: Could not find project: bcffff710dd74209b30a1d02fb2bec46. Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 28 16:08:00.742320 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 16:08:00.743836 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 16:08:00.745381 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 16:08:00.747027 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 16:08:00.749063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: bcffff710dd74209b30a1d02fb2bec46. Sep 28 16:08:00.749063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 16:08:00.749063 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1469/2945] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Sep 28 16:08:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 28 16:08:00.780038 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:00.780848 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:00.781140 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:00.819589 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] MFA Rules not processed for user `6fcc9eec4e114f538c506178c744b49a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:08:00.820693 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:00.830653 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1477/2946] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:08:00.840767 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:08:00.850707 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5946f35-e791-4a83-9cad-04206577e225 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:08:00.854988 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:08:00.855589 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:00.855873 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:00.856144 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:00.866907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:08:00.868473 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351f01d-8335-4a03-af21-e5d72aef7c36 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:08:00.880917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1470/2947] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:08:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 16:08:06.016061 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:08:06.017950 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:06.029288 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31067ae5-2d61-4e98-b46c-cc6748f8c9e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:08:06.033464 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9685e7e865604d4b9acd36902045d141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:08:06.034051 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:06.034329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:06.034633 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:06.036292 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:06.049469 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:08:06.050932 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:08:06.061657 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1478/2948] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:08:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:08:10.875314 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:08:16.056785 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=49 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:08:20.746755 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:20.747270 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:20.747387 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:20.766103 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] MFA Rules not processed for user `616d569f5bc24fb8bc5c78128ca4772f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 16:08:20.767405 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:20.774158 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1471/2949] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 16:08:21.009641 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:08:21.019520 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8f6b-3389-4126-977e-38b25aeac150 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:08:21.023103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1393d3a8c9174d08b2c6d63cf4e25b19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:08:21.023687 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:08:21.023965 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:08:21.024238 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:08:21.025961 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:08:21.037963 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:08:21.039427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:08:21.047427 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1479/2950] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:08:30.761015 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:08:31.043799 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:09:24.475558 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:09:24.477049 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:09:24.538924 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:09:24.551935 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324306-89a1-4a07-ac0b-219f5cfbbd2c service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:09:24.557334 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:09:24.557993 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:09:24.558495 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:09:24.558770 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] PATH_INFO: `/v3/users/616d569f5bc24fb8bc5c78128ca4772f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:09:24.560276 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe57d7c-dbb3-4296-aed5-7ca045d22af8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:09:24.560404 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616d569f5bc24fb8bc5c78128ca4772f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:09:24.565762 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:09:24.566103 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:09:24.566561 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:09:24.566907 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:09:24.567296 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:09:24.570130 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:09:24.584630 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:09:24.586031 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:09:24.596858 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1472/2951] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 16:09:24.621136 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616d569f5bc24fb8bc5c78128ca4772f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.628664 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616d569f5bc24fb8bc5c78128ca4772f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.629358 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616d569f5bc24fb8bc5c78128ca4772f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.630329 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616d569f5bc24fb8bc5c78128ca4772f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.638623 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616d569f5bc24fb8bc5c78128ca4772f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.655779 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1480/2952] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 16:09:24 2022] DELETE /identity/v3/users/616d569f5bc24fb8bc5c78128ca4772f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 16:09:24.829820 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:09:24.842917 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17c751-ac98-40ba-ba6e-3932f25b75d8 service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:09:24.903246 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86c62b56d134029a7eefd7d39742f82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c663536db924b2aba9e9bb0e3254711', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:09:24.903871 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:09:24.904155 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:09:24.904429 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] PATH_INFO: `/v3/projects/9c2422a69a09434a80242c7572ba9e13` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:09:24.905528 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2422a69a09434a80242c7572ba9e13)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:09:24.907496 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:09:24.954526 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2422a69a09434a80242c7572ba9e13', 'request_id': 'req-95023062-eac0-41da-b1e0-06d132eb7843'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.961745 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2422a69a09434a80242c7572ba9e13', 'request_id': 'req-95023062-eac0-41da-b1e0-06d132eb7843'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.968546 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2422a69a09434a80242c7572ba9e13', 'request_id': 'req-95023062-eac0-41da-b1e0-06d132eb7843'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.979623 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] The token cache is being invalidate because project 9c2422a69a09434a80242c7572ba9e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 16:09:24.979910 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.980538 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95023062-eac0-41da-b1e0-06d132eb7843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 16:09:24.982010 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1473/2953] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 16:09:24 2022] DELETE /identity/v3/projects/9c2422a69a09434a80242c7572ba9e13 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 16:09:34.652139 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:09:34.976511 dv-cinder-859316-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:12:09.546030 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:12:09.547560 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:12:09.692736 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06a12a-fb30-45ea-be3f-8a5b0b4d1a35 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:12:09.752225 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd633df8700344a19b2de33afb3098ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0522609f3e2487d8abf38206ef39d4e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:12:09.752866 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:12:09.753156 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:12:09.753432 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:12:09.755235 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:12:09.766302 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:12:09.767909 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b8b4c-ab56-4dba-8232-b38ce360ad88 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:12:09.778217 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1481/2954] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 16:12:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:12:19.773990 dv-cinder-859316-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}